16. AI Agent worker

16.1. What is AI Agent?

AI Agent is a versatile worker in d3VIEW that harnesses the intelligence of LLMs within the d3VIEW environment. This worker completes the user’s task as specified within the prompt using the resources that the user provides.

AI Agent worker



AI Agent worker has these inputs

  1. Prompt: User instructions on what the Agent should do.
  2. Use Sample Data: Run the Agent with sample data.
  3. Revise Prompt: Use AI to refine the user’s prompt, making it easier for the Agent to complete the desired task.
  4. Number of Iterations to revise prompt: The number of times the prompt should be revised. 1 or 0 is recommended.
  5. Tool selection: Flag on whether the Agent should be able to use the attached tools or not.
  6. Files: PDF files (PDF Only) that contain relevant information the user deems necessary to answer their prompt request

16.2. What are LLM(Large Language Models) workers?

LLM workers are components or processes that utilize Large Language Models (LLMs). In AI agent worker, LLM workers are responsible for handling specific language tasks based on prompts, collaborating with tools, and returning output to the chat interface for review.

AI Agent worker uses the intelligence contained within these LLM workers:

  1. OpenAI
  2. ClaudeAI

These workers provide the reasoning needed to complete the user’s tasks and the decision making needed to know which resources should be used and how, given said task.

L L M workers



LLM Workers

The OpenAI and ClaudeAI workers are used within the AI agent to intelligently execute user driven tasks.

Open AI worker has these inputs to define

  1. URL: URL to communicate with OpenAI servers.
  2. API-Key: API key to access OpenAI models.
  3. Model: OpenAI LLM to interact with.
  4. Role: The role that should be associated with the prompt. This is ‘user’ by default.
  5. Prompt: The message prompt that should be sent to the LLM.
  6. Files: PDF files (PDF Only) that contain relevant information the user deems necessary to answer their prompt request.
  7. Tools: Tools the model should use in a valid JSON schema format, where there is an array including the tool(s) JSONs.
  8. System message: General instructions for the model to follow while answering the prompt.
  9. Schema: If the user desires their output to be in a JSON format, they can provide a valid JSON Schema containing the format they would like it in.
  10. Tool choice: Choices for whether the LLM should use tools.
  • Auto: LLM can either use tools or respond with text, depending on what it thinks is necessary.
  • Required: LLM must use a tool for the request.
  • None: LLM cannot use any tools for the request. This is the same as simply not sending tools in the ‘tools’ input.

Parallel Tool Calls: Whether the model is allowed to call multiple tools within a single request

  1. History: Conversation history contains previous interactions between the user and LLM that the model should consider in this format

[

{

“role”: “<role>”,

“content”: “<content>”,

}, …

]

For example: If a user wants to reference a curve (“my_curve” in this example) in history to perform additional operations on the same curve, they don’t have to keep reminding the model as to which curve must be used. This curve will be stored in history.

[

{

“role”: “user”,

“content”: “Can you scale the curve: my_curve”

}, …

]

  1. Built in Tools (OpenAI only): Web search and Enable Image generation. This is an optional input and is set with a default value of “none”.
  2. Version (ClaudeAI only): Anthropic version to use when interacting with Claude models. This is an optional input and defaults to the latest Anthropic version.
  3. Max Tokens (ClaudeAI only): Number of tokens the model should be limited to generating. This is an optional input and is set with a default value of 1024.

Open AI

Open AI worker inputs


Claude AI

Claude AI worker inputs


16.3. What are AI Agent tools?

The AI agent worker integrates d3VIEW workers as ‘tools’. These tools are used to complete tasks, depending on what the user requests the AI Agent to do. For example, if the user enters the prompt “Scale the number 5 by 3”, the AI Agent realizes that it needs to use the attached ‘math_scale’ tool worker, sets the inputs accordingly, executes the worker, and responds with the results of the operation.

AI Agent tools


Execution Flow of the AI Agent worker.

  1. User specifies their desired task within the ‘Prompt’ input in the AI Agent.
  2. LLM interprets the prompt deciding what tools should be run and what inputs it should run with (if the input source is AI).
  3. Tool workers are run with their specified inputs.
  4. Worker outputs are collected and sent to the LLM for analysis.
  5. LLM creates a respond to users original response based on the outputs.

16.4. Examples to show the usage and execution of AI Agent worker.

Example 1 : Scaling the numbers

  1. AI Agent is provided with user prompt “Can you scale the numbers 10 and 20?”
  2. LLM Worker (OpenAI) Activated

The OpenAI LLM analyzes the prompt.

It recognizes that the task involves scaling numbers.

It determines that a tool worker is needed for mathematical scaling.

  1. Tool Worker Invoked: Math Scale Worker

The AI agent routes the numbers to the scale worker.

The worker applies the predefined scaling logic.

Tools worker is executed.

  1. Output Displayed
The final response is shown in the chat interface

The below video shows a simple example of scaling a number using AI Agent worker



Example 2 : Curve Derivative

  1. AI Agent is provided with user prompt “Get a stress strain curve for a “steel” material in the following array format of x,y points [{ x, y }, {x2,y2}…] and set it to the curve input in JSON stringified format and DERIVE it.”
  2. LLM Worker (OpenAI) Activated

Gets A stress-strain curve (material data for “steel”)

Formatted as a JSON array of {x, y} points

Generates a typical stress-strain curve for steel in the required format.

  1. Tool Worker Invoked: Curve Derivative

This tool takes the formatted input.

Computes the numerical derivative (e.g., dy/dx) for each segment between points.

  1. Chat Interface Displays Output

The below video shows how to get a stress strain curve for a “steel” material and derive it.



Example 3 : Drop columns from Dataset

In the below video, lets see how to drop columns from dataset using AI agent worker.



Example 4 : Selection of Metal/Polymer workflow for execution from Library using WORKFLOW_EXECUTOR as tool worker.

Here,

AI Agent is provided with user prompt.

I have a list input whose value = Metal/Polymer.

If the list input is equal to “Metal” then execute the workflow = WF1 or else execute = WF2.

Prompt for the inputs


The inputs or the Library workflows are defined in Start worker for Workflows to be selected.

Start worker inputs


Now add WORKFLOW EXECUTOR worker as tool and change the input type to AI.

Whose input will be coming from the prompt of the AI Agent worker.

Workflow executor worker


After adding the inputs and tool worker.

Execute the AI Agent worker with toggling the input to Metal and Polymer in Start worker.

  1. When list input selected as Metal the Workflow executor will select the 103 Metals postnecking calibration workflow for execution.
  2. When list input selected as Polymer the Workflow executor will select the 013 Polymer calibration workflow for execution.

Workflow executor execution


16.5. Prompt in AI Agent worker

AI Agent worker supports ‘Prompt’ input which is user instructions on what the Agent should do.

Prompts can be created manually or by using the variables from workers.

In the below video, let us see simple examples of adding prompts manuually and by using variables.