This is a guest post written by Asaf Fried, Daniel Pienica, and Sergey Volkovich from Cato Networks.
Cato Networks is introducing Secure Access Service Edge (SASE), a unified cloud-centric enterprise networking and security service that integrates Security Service Edge (SSE) capabilities such as SD-WAN, cloud networking, and firewall-as-a-service. A leading provider of (FWaaS), secure web gateways, zero trust network access, and more.
The central events page in the SASE Administration Console provides a comprehensive view of events that occur in a particular account. Because millions of events can occur within the selected time range, the goal is to filter these events using various filters until a manageable number of relevant events are identified for analysis. It’s about narrowing it down. Users can see different types of events such as security, connectivity, system, and management, each categorized by specific criteria such as threat protection, LAN monitoring, and firmware updates. However, the process of adding filters to a search query is manual and requires familiarity with the product glossary, which can be time-consuming.
To address this challenge, we recently enabled our customers to perform free text searches on event management pages, allowing new users to perform queries with minimal product knowledge. This was achieved by using a foundational model (FM) to transform natural language into structured queries that are compatible with our product’s GraphQL API.
This post shows you how to use Amazon Bedrock. Amazon Bedrock is a fully managed service that makes leading AI startups and Amazon’s FM available via API. This allows you to find the best model from a wide range of FMs. Use case. With the Amazon Bedrock serverless experience, you can get started quickly, privately customize FM with your own data, and use AWS tools to quickly integrate and deploy into your applications without managing infrastructure. can. Amazon Bedrock now powers FM with product-specific knowledge to transform free text input from users into structured search queries in product APIs, significantly improving user experience and efficiency for data management applications. I was able to do that.
Solution overview
of event This page includes a filter bar that includes both event filters and time range filters. These filters must be added and updated manually for each query. The following screenshot shows an example of an event filter (1) and a time filter (2) displayed in the filter bar (Source: Cato Knowledge Base).
An event filter is a combination of statements of the form:
- key – Field name
- operator – Evaluation operators (for example, is, in, includes, greater than, etc.)
- value – Single value or list of values
For example, the following screenshot shows a filter. Action (alert, block).
A time filter is a time range that follows the ISO 8601 time interval standard.
For example, the following screenshot shows the following time filter: UTC.2024-10-{01/00:00:00--02/00:00:00}
.
Converting free text into structured queries for event and time filters is a complex natural language processing (NLP) task that can be performed using FM. Customizing FM for specific tasks is often done using one of the following approaches:
- rapid engineering – Add instructions to the model’s context/input window to help complete the task successfully.
- Search extension generation (RAG) – Retrieves relevant context from the knowledge base based on the input query. This context is extended to the original query. This approach is used to reduce the amount of context provided to the model to only relevant data.
- Fine adjustment – Train the FM based on task-relevant data. In this case, the relevant context is not part of the input, but is embedded in the model weights.
For our particular task, we found that quick engineering was sufficient to achieve the desired results.
The event is event Since the page is product-specific, you must provide the FM with the exact steps to generate the page based on the free text query. The main considerations when creating prompts are:
- Include relevant context – This includes:
- Keys, operators, and values ​​that the model can use.
- Specific instructions. For example, numeric operators can only be used with keys that have numbers.
- Make sure validation is easy – Given the huge number of instructions and limitations, you cannot trust the model output without checking the validity of the results. For example, what if your model generates a filter with a key that’s not supported by the API?
Instead of asking FM to generate GraphQL API requests directly, you can use the following method:
- Instructs the model to return a response according to known JSON Schema validation IETF standards.
- Validate the JSON schema of the response.
- Convert this to a GraphQL API request.
request prompt
Based on the previous example, the system prompt is configured as follows:
# Genral Instructions
Your task is to convert free text queries to a JSON format that will be used to query security and network events in a SASE management console of Cato Networks. You are only allowed to output text in JSON format. Your output will be validated against the following schema that is compatible with the IETF standard:
# Schema definition
{
   "$schema": "https://json-schema.org/draft/2020-12/schema",
   "title": "Query Schema",
  "description": "Query object to be executed in the 'Events' management console page. ",
   "type": "object",
   "properties":
   {
       "filters":
       {
           "type": "array",
          "description": "List of filters to apply in the query, based on the free text query provided.",
           "items":
           {
               "oneOf":
               (
                   {
                       "$ref": "#/$defs/Action"
                   },
                   .
                   .
                   .
               )
           }
       },
       "time":
       {
           "description": "Start datetime and end datetime to be used in the query.",
           "type": "object",
           "required":
           (
               "start",
               "end"
           ),
           "properties":
           {
               "start":
               {
                   "description": "start datetime",
                   "type": "string",
                   "format": "date-time"
               },
               "end":
               {
                   "description": "end datetime",
                   "type": "string",
                   "format": "date-time"
               }
           }
       },
       "$defs":
       {
           "Operator":
           {
               "description": "The operator used in the filter.",
               "type": "string",
               "enum":
               (
                   "is",
                   "in",
                   "not_in",
                   .
                   .
                   .
               )
           },
           "Action":
           {
               "required":
               (
                   "id",
                   "operator",
                   "values"
               ),
               "description": "The action taken in the event.",
               "properties":
               {
                   "id":
                   {
                       "const": "action"
                   },
                   "operator":
                   {
                       "$ref": "#/$defs/Operator"
                   },
                   "values":
                   {
                       "type": "array",
                       "minItems": 1,
                       "items":
                       {
                           "type": "string",
                           "enum":
                           (
                               "Block",
                               "Allow",
                               "Monitor",
                               "Alert",
                               "Prompt"
                           )
                       }
                   }
               }
           },
           .
           .
           .
       }
   }
}
Each user query (added to the system prompt) is constructed as follows:
# Free text query
Query: {free_text_query}
# Add current timestamp for context (used for time filters)Â
Context: If you need a reference to the current datetime, it is {datetime}, and the current day of the week is {day_of_week}
You can also use the same JSON schema included in the prompt to validate the model’s response. This step is critical because model behavior is non-deterministic in nature and responses that do not conform to the API will break product functionality.
In addition to validating alignment, JSON Schema can also point out exact schema violations. This allows you to create policies based on different failure types. for example:
- If a field marked as required is missing, output a translation failure to the user.
- If the value specified in an event filter does not conform to the format, remove the filter, make an API request from the other value, and print a conversion warning to the user.
After FM successfully transforms free text into structured output, converting it into API requests such as GraphQL is a simple and definitive process.
To validate this approach, we created a benchmark with several hundred text queries and their corresponding expected JSON output. For example, consider the following text query:
Security events with high risk level from IPS and Anti Malware engines
This query expects the model to return the following response based on the provided JSON schema:
{
   "filters":
   (
       {
           "id": "risk_level",
           "operator": "is",
           "values":
           (
               "High"
           )
       },
       {
           "id": "event_type",
           "operator": "is",
           "values":
           (
               "Security"
           )
       },
       {
           "id": "event_subtype ",
           "operator": "in",
           "values":
           (
               "IPS",
               "Anti Malware"
           )
       }
   )
}
Define three different outcomes for each FM response.
- success:
- valid JSON
- Enabled by Schema
- Exact filter match
- partial:
- valid JSON
- Enabled by Schema
- Partial filter match
- error:
- JSON is invalid or invalid by schema
Because translation failures lead to a poor user experience, the release of this feature is conditional on reaching an error rate of less than 0.05, and the selected FMs will have a success rate that meets this criterion (i.e., a response with an exact match for the filter). percentage) was the highest.
Using Amazon Bedrock
Amazon Bedrock is a fully managed service that simplifies access to a variety of cutting-edge FMs through a single serverless API. It provides a production-ready service that can handle large requests efficiently, making it ideal for enterprise-level deployments.
Amazon Bedrock allows us to efficiently migrate between different models and makes it easy to benchmark and optimize accuracy, latency, and cost without complicating the management of the underlying infrastructure. Additionally, some vendors within the Amazon Bedrock environment, such as Cohere and Anthropic’s Claude, offer models that can natively understand JSON Schema and structured data, further increasing their applicability to specific tasks. .
We used benchmarks to evaluate several FMs on Amazon Bedrock considering accuracy, latency, and cost. Based on the results, we selected anthropic.claude-3-5-sonnet-20241022-v2:0, which met our error rate criteria and achieved the highest success rate while maintaining reasonable cost and latency. Following this, we proceeded to develop a complete solution containing the following components:
- management console – A Cato management application that users interact with to view network and security events for their accounts.
- GraphQL server – A backend service that provides a GraphQL API to access data in your Cato account.
- Amazon bedrock – A cloud service that handles hosting and serving requests to FM.
- Natural language search (NLS) services – An Amazon Elastic Kubernetes Service (Amazon EKS) hosted service that bridges the gap between the Cato management console and Amazon Bedrock. This service creates a complete prompt for FM and validates the response using JSON Schema.
The following diagram shows the workflow from a user’s manual query to extracting relevant events.
A new feature also allows users to use free text query mode. This is handled as shown in the following diagram.
The following screenshot is event The page shows free text query mode in action.
Business impact
Recent feature updates have received positive feedback from customers. Users, especially those new to Cato, find the new search functionality more intuitive and easier to navigate and navigate the system. Additionally, with the inclusion of multilingual input natively supported by FM, event This page is designed to be accessible to non-native English speakers, allowing them to interact and find insights in their native language.
One noticeable effect is a significant reduction in query time. Minutes of manual filtering are reduced and results are almost instantaneous. Account administrators using this new feature report near-zero time to value and immediate benefits with minimal learning curve.
conclusion
Accurately converting free text input into structured data is critical for applications involving data management and user interaction. In this post, we demonstrated a real-world business use case for Cato Networks that significantly improved the user experience.
With Amazon Bedrock, you now have access to a state-of-the-art generative language model with built-in support for JSON Schema and structured data. This allowed us to optimize cost, latency, and accuracy without complicating the management of the underlying infrastructure.
While the quick engineering solution met our needs, users working with complex JSON schemas may want to consider a different approach to reduce costs. Including the entire schema in a prompt can significantly increase the number of tokens in a single query. In these cases, consider using Amazon Bedrock to fine-tune your model and embed product knowledge more efficiently.
About the author
Asaf Freed He leads the data science team at Cato Networks’ Cato Research Labs. A member of Kato Control. Asaf has over six years of experience in both academia and industry in applying cutting-edge novel machine learning techniques to the networking and cybersecurity space. His main research interests include asset discovery, risk assessment, and network-based attacks in enterprise environments.
Daniel Pienica I’m a data scientist at Cato Networks with a passion for large-scale language models (LLM) and machine learning (ML). With 6 years of experience in ML and cybersecurity, he brings a wealth of knowledge to the job. With a Master’s degree in Applied Statistics, Daniel applies his analytical skills to solve complex data problems. His passion for the LLM drives him to find innovative solutions in cybersecurity. Daniel’s dedication to his field is evident in his continuous exploration of new technologies and methods.
sergei volkovich He is an experienced data scientist at Cato Networks, developing AI-based solutions in cybersecurity and computer networks. He received a master’s degree. He received his PhD in physics from Bar-Ilan University and published a paper on theoretical quantum optics. Prior to joining Cato, he held multiple positions across a variety of deep learning projects, from publishing papers on the discovery of new particles at the Weizmann Institute to advances in computer networks and algorithmic trading. Currently, his main focus is cutting-edge natural language processing.
Omer Haim He is a Senior Solutions Architect at Amazon Web Services with over 6 years of experience solving complex customer challenges through innovative machine learning and AI solutions. He has deep expertise in generative AI and container technologies, and is passionate about working backwards from customer needs to deliver scalable, efficient solutions that drive business value and technology transformation.