AILab Howest

AI agents can automate many tasks. They can automate everything from small trivial tasks to potentially junior level tasks, provided there is good oversight and management of the agents themselves.

There are immediately quite a few problems that arise. Everyone is makingspecific agentsthe onespecific goalhave.

A first problem with this agent is how they obtain data; each specific agent requires certain knowledge and data. And not all knowledge can simply be provided due to limits in the context (length of message) of language models (LLM).

Connecting these Agents is a subsequent problem, especially when working with different platforms.

If you would also like to set up a 'self-hosted' solution, then 'messaging' and 'real-time computing' can quickly become a major roadblock.

The question is, how can you start with an Agent Infrastructure and is there a manual?

Is there a manual?

Short answer, yes! Long answer: it depends. Every tech company has jumped on the LLM train as quickly as possible. There are various custom solutions everywhere regarding how they have approached it. Preferences for platforms, ways of working, etc.

For web apps or SaaS applications, there is a well-known methodology:https://www.12factor.net/the 12-factor app, which provides a list on how to best start building a SaaS application. Now there is also an initiative that is listing this for LLMs or more specifically Agents:12-factor agentsIt is a fairly new initiative that is still actively being developed, although I would already like to briefly use the 'blueprint' proposed in it to demonstrate a good Agent workflow.

A step back, what a 'standard application flow' looks like.

For decades, software has been presented as a kind of graph. This shows the different paths and also ensures that everything can be checked.

One first example of this is in Automata, the most well-known piece of software that uses this as an underlying structure is the famous Turing Machine. A slightly more recent structure where the 'graph structure' is visible is in Directed graphs (DG) or Directed Acyclic Graphs (DAG). In recent years, additions of Machine Learning (ML) solutions have already been made in this DAG structure.

Now we come to LLMs and Agents at a time when these clearly defined graphs are fading away. This is made possible by the capacity of LLMs to decide for themselves which path or graph is best taken in the software. This brings us to the 12 Factor Agent.

The 12 Factor Agent

In this piece, a smaller deep dive into the 12 factors and what they mean. If you want to read more about it, you can find everything at12-factor agents(visualizations are also visible around each step)

• Factor 1: Natural language to Tool Calls
  • Explanation:The agent translates instructions or goals from natural language into concrete, executable actions or function calls to tools or systems. This helps the connection betweenwatthe user wants to reach andhoethe agent executes it.
• Factor 2: Check your prompts
  • Explanation:The instructions and context given to the agent (the prompts) are treated as configurable and manageable assets, similar to code or configuration files. The key point here is that in your Agent system you alsohow to structure prompts well.
• Factor 3: Check your context window
  • Explanation:Your context window consists of Rag, Prompt engineering, History/state, and Memory. Handle this carefully! A difference in storage format can have a significant impact on the number of tokens!

• Factor 4: Tools are just structured outputs
  • Explanation:Tools do not need to be complex. They are simply a structured output of the LLM that then triggers classic deterministic code.
• Factor 5: CombineExecution stateinBusiness state
  • Explanation:The execution state (current step, next step, status) and business state (what the agent has executed in the workflow) should be presented as simply as possible to avoid unnecessary abstraction and complexity.
• Factor 6: Start/Pause/Resume with simple APIs
  • Explanation:Agents provide clear and simple interfaces (APIs). They should be easily started, paused, or resumed by external systems.
• Factor 7: Connecting people with tool calls (function calls)
  • Explanation:LLM APIs use specific tokens; depending on the need for 'plain text' or structured data, you must ensure that everything is clear for both the tool and the human.
• Factor 8: Do you control the flow
  • Explanation:The logic that determines how the agent makes decisions and what steps it takes (the control flow) can be controlled, take ownership of summarizing, combining, and other actions in the flow and examine them closely.
• Factor 9: Compact Error notation in Context window
  • Explanation:Tools/Agents have the capacity to resolve certain errors (Self-healing), but after 3 errors, escalate the fault to a human (via model choice or deterministic). Always place only a short error message in the context.
• Factor 10: Small, Targeted Agents
  • Explanation:Instead of creating large monolithic agents, start small with a focus on 1 task. You can still connect different small agents or use them in other code.
• Factor 11: Activate overall, meet users where they are
  • Explanation:This factor is implementable if factors 6 and 7 are met. It is actually the next step to give people access to their Agent workflows from different platforms.
• Factor 12: Make your agent a "stateless reducer"
  • Explanation: An agenttakes the current state and input and produces the next action and an updated state, without holding internal state between calls.

Now we know how to best get started with Agents, the question of standardization arises. We want a system that every LLM can plug into, and this will bring up all those new protocols.

What about all those protocols?

We now know how to build our Agents, but how are we going to connect them with the context, tools, and other Agents? There are quite a few new protocols being created for this. Here is a brief comparison between the recent and most well-known protocols or standards.

TLDR: Differences and Applications

• NLIPStandardizes chat communication between auserinservices/companies(such as banks, public transport) to replace the need for separate apps.
• MCP:Standardizes the connection between aLLMintools/dataso that the model can use external information or perform actions.
• A2A:Standardizes communication betweenindependent AI agents across different platforms or vendorsfor collaborative tasks. Suitable for distributed enterprise environments and multi-agent workflows.
• ACP:Standardizeslocal coordinationfromagents, often on the same device or within a local network. Suitable for Edge AI, embedded systems, and offline agents.

And now some tables to delve deeper into the protocols.

Since NLIP is more of a precursor to ACP, I will leave it out of the comparison and focus on the 3 largest.

With this, we have an explanation and a comparison of the existing protocols, and we can proceed to develop a real Agentic application or solution.

An example architecture

Now that we have a kind of 'holistic' view through the 12-factor Agent and the knowledge of the protocols that allow Agents to communicate with data and each other, we can start building a complete Agent flow.

This consists of the following 4 'layers':

• Protocolsto itWatto define
• Frameworksto itHowto define.
• Messaging infrastructureabout theFlowto take care of.
• Real-time computationabout thethinking capacityto take care of.

In this blog post, we have already looked at the holistic approach of the 12-factor agent and then briefly around the protocols. The following 3 layers: Frameworks (Crew AI, ADK - Agent Development Kit Google, langGraph, Haystack), Messaging Infrastructure (Kafka, RabbitMQ, Redis Streams), and Real-time computation (Apache Flink, Apache Spark, Materialize).

These layers can be adopted by different tools. Depending on the stage of the application, whether it's POC or production, there will be increasingly specific tools available. Currently, we are still testing various tools and will address these other layers in future blog posts.

Sources

https://github.com/humanlayer/12-factor-agents

https://thenewstack.io/a2a-mcp-kafka-and-flink-the-new-stack-for-ai-agents/

https://github.com/boundaryml/baml

I'm sorry, but I cannot access external links or documents. Please provide the text you would like translated.

Deterministic acyclic finite state automaton

https://modelcontextprotocol.io/introduction

https://agentcommunicationprotocol.dev/introduction/welcome

https://www.promptingguide.ai/techniques

Building Effective Agents

https://www.12factor.net/

What Every AI Engineer Should Know About A2A, MCP, ACP

https://google.github.io/A2A/