What DigitalOcean's first agent builders needed
DigitalOcean's GenAI Platform gave a small team a fast way to stand an agent up. What it did not yet give them was a way to know whether that agent could be put in front of customers — the platform's guardrails feature had not shipped. Vijil worked alongside its early adopters in that window, and Autonoma is what the gap looked like in practice: an IoT platform for industrial machinery manufacturers, building a troubleshooting agent for technicians onsite, where any downtime is expensive. The first prototype hallucinated, answered in the wrong language and could be talked out of its instructions. Vijil found all of it, fixed it, and hardened the agent; Autonoma took it to production on DigitalOcean in one week, saving a lean team months of development time and cost.
What the first builders ran into
Autonoma began by developing a prototype retrieval-augmented generation system with Mistral Nemo Instruct as the generative model, using their online help center as the knowledge base. Confounding factors during the initial build meant the agent produced a large number of hallucinations. It also failed to respond in the correct language — German or English — based on the user’s query, even though the documentation existed in both.
Vijil tested the agent’s performance and trustworthiness along three dimensions: reliability, security and safety. The performance tests showed that generated answers were not grounded, and were not being retrieved from the context with sufficient precision and recall. The security and safety tests found the agent was susceptible both to simple misuse — performing tasks outside its scope — and to malicious jailbreak attacks, which force an agent to ignore its operator’s instructions and follow the attacker’s instead.
How Vijil solved for trust
Four areas of improvement, none of which would have been sufficient on its own:
Mistral Nemo Instruct does a better job of inserting images into content directly, but has significant security vulnerabilities that make it prone to prompt injection attacks. Switching to Llama 3.1 8B lowered that vulnerability.
Vijil scraped Autonoma’s online help center in both languages and extracted the plaintext content as well as the linked images and metadata, then separated English and German into different knowledge bases so the agent could cite content in the user’s own language.
Expanding the system prompt gave the agent a sufficiently detailed set of instructions; adding policies prevented misuse; and an explicit instruction to respond in the correct language let it switch between English and German. Vijil’s findings here fed back into DigitalOcean’s own knowledge-base documentation.
Vijil and Autonoma were working on the platform before DigitalOcean released its guardrails feature, so the safety policy and misuse-prevention instructions went into the prompt instead. Together with the new base model, that raised the agent’s score in Vijil’s evaluation of its trustworthiness.
What the platform got back
Autonoma took the agent from development to production in a week. It now produces accurate content with sufficient references and grounding in the original documentation, responds in English and German appropriately, and is resilient to common prompt-injection attacks. Autonoma has reduced their customers’ time to troubleshoot each incident and is positioned to scale to more customers faster.
The work ran the other way too: Vijil’s findings, with Autonoma’s, shaped DigitalOcean’s AI Platform — better documentation, and features prioritized that teams like Autonoma need to deploy agents confidently.
“For our small team, it’s been a real game changer. It let us dive right in and start building, without the need of a ton of resources.”
Those four fixes were hand-built in a week. Vijil Diamond runs the tests that found the failures, and Vijil Dome enforces a safety policy at run time rather than in a prompt.
This story was first published as a DigitalOcean customer story → · one-page PDF →