Taipei Zoo Course Assistant LINE Bot

Taipei Zoo runs environmental-education activities every month: keeper talks, themed stations, scheduled courses. The information is spread across several pages on the zoo's website, and ticket prices, closures and transport live on other pages again. A visitor planning a day out with a child or an elderly parent has to piece together what is on that day, what the tickets cost and which exhibits are closed.

I wanted people to simply ask in LINE, in their own words. The real problem turned out not to be how to wire up GPT, but which questions the model should answer and which it must not. "What is on Saturday" and "how much is a student ticket" have one right answer, and a single wrong reply destroys trust. "How do we visit without tiring grandma out" has no single answer, and that is where a language model earns its place.

The primary users are ordinary visitors: parents, teachers, family members accompanying older relatives, all asking through a LINE official account. They do not know how the website is structured, they phrase things casually, and they often spread one request over several messages ("three of us need tickets", "one is a Taipei resident", "and a student").

The second user is me. The bot doubles as a portfolio piece that strangers can try, so it has to survive abuse without running up the bill, and it has to keep its timetable current while nobody is watching.

• No labelled data. The original plan had a home-trained BERT for interest classification, but there were no real conversations to label, so any F1 score would have been a projection.
• Cost. Every GPT and embedding call costs money. Off-topic questions should cost nothing, and neither should questions with a deterministic answer.
• Data goes stale. The timetable changes monthly and the open-data format drifts a little each month; manual upkeep once lapsed for half a year without anyone noticing.
• One person operating it. One Cloud Run service and one Firestore database; no room for a vector database or a PostgreSQL instance to look after.
• Public exposure. Strangers can try it, so there is a per-user message cap, and every external dependency must fail open rather than turn into a 500 on the webhook.

One rule drives the architecture: decide whether the question has a single correct answer before deciding whether to let the language model improvise.

• Deterministic questions (tickets, hours, closures, transport, what is on a given day) are structured into CSVs and answered by Python rule matching; GPT never touches the content. Group ticket quotes follow the same split: GPT only extracts who is what, Python does the arithmetic.
• Open-ended questions (itineraries, exhibit descriptions, course suggestions) go through cosine retrieval over a local embedding index first, then gpt-4o-mini phrases the answer and tags the interest level. No whole-section dumping, no external vector database.
• Off-topic questions (stocks, code, maths) meet a three-layer guard, cheapest first: a keyword blacklist, a similarity threshold against the index, and a lightweight LLM check with conversation history only for the grey zone. Thresholds were calibrated on real production questions.
• Memory and reminders live in Firestore: five rounds and seven minutes of short-term memory, one reminder per interest event, everything wrapped to fail open.
• Dropped: the home-trained BERT (no data), APScheduler push notifications (replaced by a reminder appended to the next reply), PostgreSQL (containers have no persistent disk).

Two AI collaborators worked on this project: Cursor for the February 2026 scaffold and environment fixes, Claude Code for the RAG, Firestore, blind-testing and operations work from September on. My job stayed the same: decide who a given problem belongs to, then turn that call into a requirement that can be checked.

The clearest example is that I turned the test method itself into a Claude Code skill. Hand-written test cases favour questions I already know the bot can answer, so they never find the gap between what a visitor expects and what the bot can do. The skill requires that I supply only a persona (a relative pushing a wheelchair, a teacher planning a field trip); a sub-agent that has never seen the data folder does the asking, writes every question before asking any, and classifies only after the last reply. Replies come back from Cloud Logging in four classes: answered well, expected an answer but got none, half an answer, correctly declined.

Two rounds and 35 questions surfaced three problems no pre-designed test would have caught: a five-message cap that stopped the service at question six, the word for "rest" read as a closure query, and a group-ticket flow that would not let go. The other key collaborations followed the same pattern: I brought an observation (a failing build log, a 21-hour crash loop, a timetable three weeks stale) and a requirement; the AI found the root cause and implemented the fix, with the reasoning kept in the code's docstrings.

• The bot is live on Google Cloud Run, deployed automatically from main by Cloud Build, and reachable through a LINE official account.
• Six development stages and two architecture-level turns (rules first, then RAG plus memory), all traceable to commits and a written retrospective.
• Ticket, closure, opening-hour and course-date queries no longer pass through GPT's judgement; off-topic questions are stopped before GPT is called.
• The timetable rebuilds itself daily from the zoo's open data, with a quality re-check and a liveness check, so a silently stalled pipeline is now visible.
• The home-made blind-testing skill ran 2 personas and 35 questions and led directly to 3 fixes; a RAG golden test set and a locust load test each caught a class of problem that sequential testing could not.

There is no accuracy or F1 figure for this project. What there is: a reason for every decision, a source for every fix, and a repeatable test method whose rule is that the person asking must not know the answers.

In hindsight the two architecture turns were the same sentence, but I only articulated it clearly the second time. Starting over, I would sort the data into "has a right answer" and "does not" on day one and design the architecture from that split, instead of handing everything to GPT and clawing it back one failure at a time.

Second, a fail-safe has to be finished. The timetable pipeline was built to skip safely on error, but nobody would know it had been skipping for weeks, so the silent failure merely shrank from six months to three weeks. Failing safely and making the failure visible are two separate jobs, and both have to be done.

Third, blind testing should have started earlier. The problems it found (a cap set too low, a keyword false positive, a stuck flow) were easy to fix; the hard part was having someone who did not know the answers ask before there were real users. Now that the tool exists, it belongs after every major change, not as an afterthought long after launch.