Common Workflows

Building an agent from scratch#

You: Create a new tier-1 support agent called "RoomMedic" in workspace 5

AI: I'll create that agent for you.
[Calls create_app with name="RoomMedic", workspaceId=5]
Created agent "RoomMedic" with ID 847.

You: Set the system prompt to be a Crestron tier-1 support tech that
walks field techs through TSW touch panel and Cresnet troubleshooting,
and escalates anything that needs a programmer.

AI: I'll update the system prompt.
[Calls update_system_prompt]
Updated system prompt for RoomMedic.

You: Add the Crestron docs site as a knowledge source:
https://docs.crestron.com

AI: I'll add that URL as a knowledge source with default crawl depth.
[Calls add_url_source with url="https://docs.crestron.com"]
Added knowledge source. Status: PROCESSING.
It will be ready in a few minutes.

You: Test it with "TSW-1070 shows offline. What do I check first?"

AI: I'll send that test message.
[Calls chat_with_app with message="TSW-1070 shows offline. What do I check first?"]

Agent response: "Start at the bus. Confirm Cresnet power on the panel,
then verify the panel's IP via Toolbox..."

Bulk updating agents#

You: List all my agents

AI: [Calls list_apps]
Found 12 agents:
- ID 123: Acme Corp Support
- ID 124: Northshore Hospital AV
- ID 125: Riverside University Classrooms
...

You: Update every agent that's still using GPT-3.5 to use GPT-4o-mini.

AI: I'll check each agent's model and update the ones using GPT-3.5.
[Calls get_app for each, then update_model where needed]
Updated 7 agents from GPT-3.5 to GPT-4o-mini:
- Acme Corp Support (123)
- Northshore Hospital AV (124)
- Riverside University Classrooms (128)
...

You: Update all 50 client-specific support agents to use the new escalation
template, but space out the requests to stay under the rate limit.

AI: I'll update agents with a 2-second delay between calls
to stay within the BUILDER tier limit of 30 requests/minute.
[Updates agents one by one with delays]
Progress: 25/50 agents updated...

Migrating knowledge sources#

You: List knowledge sources in my master AV docs agent (ID 100)

AI: [Calls list_knowledge_sources]
Found 8 knowledge sources:
1. docs.crestron.com (URL, 45 pages)
2. Biamp Tesira Programming Guide.pdf (FILE)
3. QSC Q-SYS Designer Help Export.pdf (FILE)
...

You: Add all the same URLs to my new client agent (ID 200)

AI: I'll add each URL source to the new agent.
[Calls add_url_source for each URL]
Added 5 URL sources to agent 200:
- docs.crestron.com
- support.biamp.com
- q-syshelp.qsc.com
...

Note: file sources need to be re-uploaded. I can't access the original
file content from this side. Or you can use link_knowledge_source to
share the same file source between agents without re-uploading.

Setting up analytics monitoring#

You: Show me analytics for my main support agent (ID 123) for the past week

AI: [Calls get_app_analytics with since="7 days ago"]

Analytics for Acme Corp Support (last 7 days):
- 1,247 conversations
- 3,891 messages
- 312 unique users
- 4.2/5.0 average satisfaction
- 89% resolution rate

You: What are the most common questions?

AI: [Calls get_top_queries]
Top queries:
1. "Crestron TSW unresponsive" (142 times)
2. "Biamp Tesira no audio in Boardroom 4" (98 times)
3. "How do I bring the headend back up?" (87 times)
4. "Schedule a service call" (76 times)
...

You: Find conversations where techs hit a dead end ("I'm stuck",
"this isn't working", "need a programmer").

AI: [Calls search_conversations with query="stuck not working need programmer"]
Found 23 conversations with escalation signals:
- Session abc123: Field tech stuck on Cresnet bus power...
- Session def456: Tech couldn't resolve Q-SYS DSP routing...

Creating and running evaluations#

You: Create an evaluation for my Crestron support agent based on a recent
good support conversation.

AI: First, let me find a recent successful conversation.
[Calls search_conversations]
Found a conversation with clear question/answer pairs from yesterday.

[Calls get_conversation to read the full messages]
I'll extract test cases from those exchanges and create an eval.

[Calls create_eval with name="Crestron Tier-1 Quality" and 8 test cases]
Created evaluation "Crestron Tier-1 Quality" with 8 test cases.
Want to review them?

You: Run the evaluation

AI: [Calls run_eval]

Evaluation complete. Results:
- Overall: PASSED (7/8 tests)
- Pass rate: 87.5%

Failing test:
- Test 7: Cresnet bus power loss across multiple devices
  Expected: walk through bus power verification, then DM-NVX network check
  Got: generic "restart the processor" response

You: Update the system prompt to better handle Cresnet bus issues,
then run the eval again.

AI: [Calls improve_system_prompt with focus="Cresnet bus troubleshooting"]
[Calls run_eval]
All 8 tests now pass.

Managing tags at scale#

You: Create a tag called "truck_roll_needed" for my support agent
to detect when a remote fix isn't possible and a tech has to go on site.

AI: [Calls create_tag]
Created tag "truck_roll_needed". Want me to generate training utterances?

You: Yes, generate 200 utterances for "this needs a tech on site"
in an AV service context.

AI: The tag is now trained. Sample utterances:
- "I can't reach the headend, the rack room is locked"
- "The processor is bricked, can't get into Toolbox"
- "We need someone here to swap the projector lamp"
- "Touch panel won't power on at all, even after a hard reset"
...

You: Show me the last 10 messages that matched this tag.

AI: [Calls list_tag_instances]
Recent truck-roll matches:
1. "Display still won't power on, we tried everything" - 2 hours ago
2. "Need someone to recommission the DSP, lost config" - 5 hours ago
3. "Conf room mic array is dead, need a replacement" - yesterday
...

Deployment management#

You: Show me all deployments for my support agent.

AI: [Calls list_slack_deployments, get_email_deployment,
get_whatsapp_deployment, get_voice_config]

Acme Corp Support deployments:
- Slack: connected to "Acme AV" workspace
  - Channels: #av-support, #project-rooms
  - Last message: 2 hours ago

- Email: enabled
  - Address: avhelp@acme.avcodex.com
  - Last email: yesterday

- WhatsApp: not configured

- Voice: enabled
  - Phone: +1 (555) 123-4567
  - Language: English
  - Last call: 3 days ago

You: Update the voice config to also support Spanish (we have field
techs in CDMX).

AI: [Calls update_voice_config with language="es,en"]
Updated voice configuration. The agent now handles English and Spanish
phone calls.

Prompt engineering workflow#

You: My field tech support agent gives overly long responses.
Help me make it more concise. Field techs are usually on a phone in a rack room.

AI: Let me analyze your current prompt and suggest improvements.
[Calls get_system_prompt]
[Calls improve_system_prompt with focus="conciseness"]

Updated prompt changes:
1. Added "Keep responses under 3 sentences when possible".
2. Added "Lead with the action, not the explanation".
3. Removed redundant "be helpful" phrasing.

Want to test it?

You: Yes, test with "TSW-1070 shows offline".

AI: [Calls chat_with_app]

Before: "Thanks for reaching out about your Crestron TSW-1070! I'd be
happy to help you troubleshoot why your touch panel is showing offline.
There are several things we can check together. The first thing I'd
recommend is verifying the Cresnet bus power and confirming the panel
is receiving power..."

After: "Check Cresnet bus power first. Then confirm the panel's IP via
Toolbox. If both look good, check the program for the device ID."

Field-tech friendly.

Multi-agent analytics dashboard#

You: Give me a summary of all my agents' performance this week.

AI: [Calls list_apps]
[Calls get_app_analytics with since="7 days ago" for each]

Weekly performance summary (12 agents):

Top performers:
1. Acme Corp Support: 1,247 conversations, 4.5 stars
2. Northshore Hospital AV: 432 conversations, 4.3 stars
3. Riverside University Classrooms: 298 conversations, 4.1 stars

Needs attention:
- Hawthorne School District AV: 2.8 stars (down from 3.5)
- Pinegrove Manufacturing: only 12 conversations (was 89 last week)

Total across all agents:
- 2,891 conversations
- 8,234 messages
- 892 unique users

You: What's wrong with the Hawthorne School District agent?

AI: [Calls search_conversations for the Hawthorne agent]
[Calls get_top_queries]

Found it. Hawthorne rolled out new Q-SYS Core Nano DSPs across their
district last week. The agent has the old Crestron-only docs but not
the Q-SYS reference.

Top unanswered queries:
1. "How do I push a Q-SYS design to the Core Nano?" (34 times)
2. "Where do I find Q-SYS Lua sample code for room control?" (28 times)
3. "Q-SYS Reflect logs show DSP offline" (19 times)

Recommendation: add the Q-SYS Designer Help Export PDF and the QSC Lua
reference docs as knowledge sources.

Quick reference: common prompts#

"List all my agents"
"Create an agent called [name] in workspace [id]"
"Delete agent [id]"
"What model is agent [id] using?"

"Add [url] to agent [id]'s knowledge base"
"List knowledge sources for agent [id]"
"Refresh all knowledge sources for agent [id]"

"Test agent [id] with message '[message]'"
"Run all evaluations for agent [id]"
"Create an evaluation from conversation [session_id]"

"Show analytics for agent [id] for the past week"
"Find conversations mentioning '[keyword]'"
"What are the top questions techs ask agent [id]?"

"Create a tag called '[name]' for agent [id]"
"Show messages that matched tag '[name]'"

Next steps#