If you’ve already identified strong use cases for AI, you’ve probably evaluated different solutions in the market. Chances are, no prepackaged AI solutions meet what your organization needs to drive real transformation. ERP providers’ AI solutions barely skim the surface and niche AI solution providers don’t understand the nuances of your business.
The natural next question is, “Should we build it ourselves?” This might seem like a massive undertaking. But training organization-specific AI models in a secure environment might be the best way to get the most out of AI and see tangible results.
Implementing an AI solution is more than just putting a large language model to use. Several basic components go into building a successful AI solution for your company. These are outlined below to clarify what it takes to truly implement effective AI.
This is the heart of the generative AI solution. This is where you would build models that are the “brains” of the AI platform. The LLM provides the base language understanding and generation capabilities. The AI model receives the input, processes the information, and produces the outputs.
For example, a manufacturing company may build an advanced planning and scheduling AI model that considers all production constraints, product alternatives, delivery dates, production timelines, bill of materials, and labor supply and automatically creates optimized production schedules based on all these factors.
Options to consider:
Tools like ChatGPT or Gemini in an “open environment” don’t always provide consistent answers. Orchestration frameworks ensure your LLM is retrieving the right information and includes all the variables needed. These frameworks simplify the process of loading data, splitting documents, creating embeddings, and orchestrating retrieval and generation (RAG). The orchestration model automates repetitive steps in AI model processing.
In the manufacturing example from above, the orchestration framework would manage the steps in the AI model. These steps may include gathering backlog production from the ERP, calculating inventory on hand, retrieving sales forecasts from the CRM, and so on. The orchestration manages all steps so they’re followed in the right order and none are skipped.
Options to consider:
These transform text into numerical vectors for semantic similarity searches. Vector embeddings are an automated way to tag unstructured data as numerical vectors (or sets of attributes). This teaches the AI model what is relevant in documents and images and tags the attributes to the respective document or image.
In the same manufacturing example, a series of documents with custom product diagrams and specifications may need to be considered in the production planning process. The AI model would need to learn and tag what items are relevant to which decisions. A product diagram may have a series of attributes like material thickness, length, strength, color, etc. The vector embedding tags and indexes produce diagrams with associated attributes that can be understood by AI models. These attributes or embeddings are a part of fine tuning the AI solution. It’s helpful for leveraging information within PDFs, JPEGs, Word, Excel, etc.
Options to consider:
The vector database is the means for storing the large volume of vector embeddings. It allows for a quick search and real-time RAG (retrieval augmented generation). The vector embeddings go beyond a simple keyword search. Instead, they identify things like similar shapes, patterns, and colors. The patterns could be similarities (or anomalies) in data in ERP systems, product diagrams, customer attributes, or production performance.
For example, a vector search for “product diagram” won’t just retrieve instances of the words “product diagram.” It will also retrieve product manuals containing pictures of engineering schematics. In the manufacturing example, product diagrams may automate bill of material generation for material requirements planning.
Options to consider:
Your organization will likely want to control and secure AI infrastructure and maintain ultimate flexibility. It’s possible to host LLM embedded generation and vector searches in the cloud, but predictability and availability of information processing is unknown. With local hardware, teams can directly monitor and optimize the AI environment and upgrade components as needed. There are fewer concerns about latency or reliance on the tools specific to the cloud hosting provider.
Furthermore, cloud charges could be unpredictable when entering the new territory of generative AI. The last thing you need are surprise cloud charges for a model you’re testing with unknown variables.
Options to consider:
Organizations with traditional development teams may already have this in house. However, observability and monitoring tools for AI development require additional considerations compared to traditional software development because AI systems are more complex, involve continuous learning, and can degrade over time in ways traditional software doesn’t. This includes bias detection, performance degradation, data quality issues, and cost monitoring. AI model monitoring often identifies the need for re-training. To support this, logging and monitoring frameworks should be able to handle streaming data, detect anomalies, and track performance metrics for LLM inference.
In the manufacturing example, suppose the production department implements a new process to reduce machine changeover time between production runs, but the AI model sees it as an anomaly, not a trend. The AI model must be refined to understand the cycle time reduction is a new baseline. Without observability and monitoring, AI models may drift in the wrong direction.
Options to consider:
These make up the front-end interfaces where people interact with the AI solution. The user interface could be a simple chatbot or a series of prompts to respond to. Depending on the complexity of the man-machine interface required, options might include pre-packaged frameworks for custom web applications.
In the manufacturing example, user interface prompts could run a production schedule based on date options, with workflows approving it for manufacturing execution systems.
Options to consider:
Keep in mind, AI evolves fast. New models, techniques, and frameworks are constantly emerging. Stay ahead by continuing to learn and stay up to date with industry-specific AI developments. Regularly revisit the overall AI strategy and be prepared to adapt systems and processes accordingly.
At Trenegy, we help organizations re-envision how to create and deliver value through AI. To chat more, email us at info@trenegy.com.
Trenegy is a non-traditional consulting firm based in Houston, Texas.