Impact of AI on Developer Productivity: Faster development cycles: Code suggestions and automation reduce time spent on repetitive tasks. Improved code quality: AI tools identify bugs or security risks that may go unnoticed by manual reviews. Enhanced learning: Engineers can receive real-time feedback or even ask AI for code explanations to learn new patterns or frameworks.

GitHub Copilot is a game-changer for writing code. Powered by OpenAI’s Codex model, Copilot suggests lines of code based on the context of what you’re writing. It’s especially useful when you’re working with repetitive tasks or writing boilerplate code.

ChatGPT, an AI chatbot developed by OpenAI, is not just a tool for casual conversations. It can be used to ask technical questions, explain difficult code, or even generate ideas for solving specific coding problems. Developers often use it for quick consultations — whether it’s about debugging or understanding the intricacies of a particular algorithm.

AI-Assisted System Architecture and Design As AI becomes more sophisticated, it may start to play a role in designing system architectures. Currently, system design is one of the more complex tasks that engineers handle, requiring a deep understanding of the trade-offs between different architectural patterns (monolithic vs. microservices, synchronous vs. asynchronous communication, etc.). Future AI tools could help design optimal architectures by analyzing the specific needs of a project, performance goals, and scalability requirements. AI could suggest which patterns, frameworks, or technologies are best suited for a given application. It could even generate architecture diagrams, API designs, or database schemas based on historical data from similar projects. This would revolutionize system design, making it faster and more accessible to engineers of all levels. While experienced architects would still be needed to make judgment calls, AI could drastically reduce the time spent on initial design phases, especially in large and complex systems.

What’s an OpenAI Assistant? Think of it as a software glue that affords you to gel together agent-like capabilities in your applications to conduct tasks expressed as instructions in natural language to an Assistant. Able to understand instructions, it can leverage OpenAI’s SOTA models and tools to carry out tasks. With Assistants stateful API, you can create Assistants within your application, providing you access to three types of supported tools: Code Interpreter, Retrieval, and Function calling [5]. At the core it has few concepts and components that cogently interact together, to enable agent-like capabilities.

Assistants API, concepts, components, and tools Unfortunately, OpenAI documentation falls short in explaining or illustrating these components into finer details and showing how they work together. Randy Michak of Empowerment AI does a fine job of dissecting these core components and illustrating their flow and data interactions [7]. Inspired by Michak, I mildly modified Figure 4, showing dynamic interaction and data flowing among Assistants API components.

To get started with Assistants, the OpenAI guide stipulates four simple steps to use the Assistants API to glue together these core components for coordination [8]. Step 1: Create an Assistant, to declare a custom model and provide instructions for the Assistant. This helps the Assistant to elect the appropriate supported tool to employ. Step 2: Create a Thread, a stateful session for the Assistant to retrieve messages from and add Assistant messages to. Step 3: Use the Thread as a conversational session to add messages for the assistants to consume. Step 4: Run the Assistant on a newly added Thread message to trigger responses. This run is Assistant’s asynchronous runtime environment.

How does it all work together?

Let’s methodically walk through a simple example where we want to accomplish the following: Integrate Assistants API, using Retrieval tool, to a) upload a couple of pdf documents and b) use an Assistant to query the contents of the document. Consider this as a mini Retrieval Augmented Generation (RAG) application. Use Files objects to upload the pdf files so that the Assistant can access them. Create and employ the Assistant, Threads, Messages, and Run objects to query the uploaded pdf documents. Coordinate all these concrete objects to interact and interplay together as part of my application.

Step 1: Create File objects as our knowledge base Upload your PDFs in the retrievers’ database, using a File object. The Assistants API breaks them into parts, as chunks, and saves them, as indexes and vector embeddings. When you ask a question, Retrievers find the best matches and help the Assistant give you a detailed answer, just like a big RAG retriever.

Step 2: Create an Assistant object. To use an Assistant and conduct tasks, first, create an AI Assistant object. Supply the Assistant with a model, instructional behavior, tools to use, and file IDs to employ for its knowledge base, as parameters.

Both OpenAI programing guide and Anyscale Endpoints blog [7] distill down to simple steps: Call the model with the user query and a list of functions defined in the Chat Completions API parameter as tools. The model can choose to call one or more functions; if so, the content will be a stringified JSON object adhering to your custom schema. Parse the string into JSON in your code, and call your function with the provided arguments if they exist. Call the model again by appending the function response as a new message, and let the model summarize the results back to the user. Following the above simple steps, our user_content to the LLM generates three required parameters (location, latitude, longitude) as a JSON object in its response.

Examples and Use Cases of Function Calling in LLM

Apart from the above use cases mentioned in the Open AI programming guide [10], Ben Lorica visually and comprehensively captures use cases of general function calling in LLMs, including the OpenAI Assistant Tools API [11]. Lorica succinctly states that early use cases include applications such as customer service chatbots, data analysis assistants, and code generation tools. Other examples extend to creative, logistical, and operational domains: writing assistants, scheduling agents, summarizing news., etc.