APIs

The OpenAPI Specification Explained The OpenAPI Specification is the ultimate source of knowledge regarding this API description format. However, its length is daunting to newcomers and makes it hard for experienced users to find specific bits of information. This chapter provides a soft landing for readers not yet familiar with OpenAPI and is organized by topic, simplifying browsing. The following pages introduce the syntax and structure of an OpenAPI Description (OAD), its main building blocks and a minimal API description. Afterwards, the different blocks are detailed, starting from the most common and progressing towards advanced ones. Structure of an OpenAPI Description: JSON, YAML, openapi and info API Endpoints: paths and responses. Content of Message Bodies: content and schema. Parameters and Payload of an Operation: parameters and requestBody. Reusing Descriptions: components and $ref. Providing Documentation and Examples: description and example/examples. API Servers: servers.

Introduction to OpenAPI Overlay Specification The Overlay Specification defines a document format for information that transforms an existing OpenAPI description yet remains separate from the OpenAPI description’s source document(s). The Overlay Specification defines a mechanism for providing consistent, deterministic updates to a given OpenAPI description, as an aid to automation throughout the API lifecycle. An Overlay can be applied to an OpenAPI description, resulting in an updated OpenAPI description. OpenAPI + Overlays = (better) OpenAPI One Overlay might be specific to one OpenAPI description, or general enough to be used with multiple OpenAPI descriptions. Equally, one OpenAPI description pipeline might apply different Overlays during the workflow.

Use cases for Overlays Overlays support a range of scenarios, including: Translating documentation into another language Providing configuration information for different deployment environments Allowing separation of concerns for metadata such as gateway configuration or SLA information Supporting a traits-like capability for applying a set of configuration data, such as multiple parameters or multiple headers, for targeted objects Providing default responses or parameters where they were not explicitly provided Applying configuration data globally or based on filter conditions

Resources for working with Overlays The GitHub repository for Overlays is the main hub of activity on the Overlays project. Check the issues and pull requests for what is currently in progress, and the discussions for details of future ideas and our regular meetings. The project maintains a list of tools for working with Overlays.

Example: Add multiple parameters to selected operations One of the most requested features for OpenAPI is the ability to group parameters and easily apply all of them together, to either some or all operations in an OpenAPI description. Especially for common parameters that always come as a set (pagination or filter parameters are a great example), it can be more maintainable to use them as a “trait” and apply the set as part of the API lifecycle rather than trying to maintain a source of truth with a lot of repetition. This approach leads to good API governance, since if the collection of fields changes then the update is consistently applied through automation. In the following example, any operations with the extension x-supports-filters set to true will have two inline parameters added to their parameter collection, and an x-filters-added tag for decoration/debugging.

Example: add a license Every API needs a license so people know they can use it, but what if your OpenAPI descriptions don’t have a license? This example shows an Overlay that adds a license to an OpenAPI description. Here’s the Overlay file, with just one action to add or change the info.license fields: overlay: 1.0.0 info: title: Add MIT license version: 1.0.0 actions: - target: '$.info' update: license: name: MIT url: https://opensource.org/licenses/MIT You can use this Overlay with different OpenAPI files to make the same change to a batch of files.

Example: tag DELETE operations To add the same tag to all operations in an OpenAPI description that use DELETE methods, use an Overlay like the example below. This example adds an x-restricted tag to all delete operations: overlay: 1.0.0 info: title: Tag delete operations as restricted version: 1.0.0 actions: - target: $.paths.*.delete update: tags: - x-restricted This overlay adds x-restricted to the tags array for each delete operation. If the tags array doesn’t exist, it’ll be created; if it does, the new tag is added to the existing array. You can use an approach like this to make other changes to all matching operations.

Resolve References A reference is said to be resolved within a tool if: Its target has been identified Any modifications to the target required by the OAS (e.g. because of fields adjacent to "$ref") have been performed on a copy of the target The resulting target value has been associated with the reference source in some way that the tool can easily use when needed A reference is said to be removed if it has been replaced by its (possibly modified) target. Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it. In many cases this is not significant, except that not knowing how the parsed OAD relates to the references in your JSON or YAML document may make debugging more difficult. While plain JSON documents form a tree structure, an OpenAPI Description with resolved references is not necessarily a tree, but a graph. Tools that resolve references in-memory and work with the graph structure can process all OADs. Tools that rely on reference removal, either as part of the tool or by a separate pre-processing tool, can only support OADs that form trees.

What are references? A reference is a keyword and value that identifies a reference target with a URI. In some cases, this URI can be treated as a URL and de-referenced directly. In other cases, as we will see in the (forthcoming) guide to resolving references, it is helpful to separate the target’s identity from its location. External references are how multiple documents are linked into a single OpenAPI Description (OAD). This means that referencing impacts how other linkages, such as those that use Components Object names, or values such as operationId in the Path Item Object, work. These other linkages can only work if the document (or with many tools, the specific JSON object) containing the name or other identifier has been referenced.

A taxonomy of references References exist in several variations in the OpenAPI Specification (OAS) versions 3.0 and 3.1, as shown in the following table. Note that an adjacent keyword is a keyword in the same JSON Object (whether it is written in JSON or YAML) as the reference keyword.

What’s an OpenAI Assistant? Think of it as a software glue that affords you to gel together agent-like capabilities in your applications to conduct tasks expressed as instructions in natural language to an Assistant. Able to understand instructions, it can leverage OpenAI’s SOTA models and tools to carry out tasks. With Assistants stateful API, you can create Assistants within your application, providing you access to three types of supported tools: Code Interpreter, Retrieval, and Function calling [5]. At the core it has few concepts and components that cogently interact together, to enable agent-like capabilities.

Assistants API, concepts, components, and tools Unfortunately, OpenAI documentation falls short in explaining or illustrating these components into finer details and showing how they work together. Randy Michak of Empowerment AI does a fine job of dissecting these core components and illustrating their flow and data interactions [7]. Inspired by Michak, I mildly modified Figure 4, showing dynamic interaction and data flowing among Assistants API components.

To get started with Assistants, the OpenAI guide stipulates four simple steps to use the Assistants API to glue together these core components for coordination [8]. Step 1: Create an Assistant, to declare a custom model and provide instructions for the Assistant. This helps the Assistant to elect the appropriate supported tool to employ. Step 2: Create a Thread, a stateful session for the Assistant to retrieve messages from and add Assistant messages to. Step 3: Use the Thread as a conversational session to add messages for the assistants to consume. Step 4: Run the Assistant on a newly added Thread message to trigger responses. This run is Assistant’s asynchronous runtime environment.

How does it all work together?

Let’s methodically walk through a simple example where we want to accomplish the following: Integrate Assistants API, using Retrieval tool, to a) upload a couple of pdf documents and b) use an Assistant to query the contents of the document. Consider this as a mini Retrieval Augmented Generation (RAG) application. Use Files objects to upload the pdf files so that the Assistant can access them. Create and employ the Assistant, Threads, Messages, and Run objects to query the uploaded pdf documents. Coordinate all these concrete objects to interact and interplay together as part of my application.

Step 1: Create File objects as our knowledge base Upload your PDFs in the retrievers’ database, using a File object. The Assistants API breaks them into parts, as chunks, and saves them, as indexes and vector embeddings. When you ask a question, Retrievers find the best matches and help the Assistant give you a detailed answer, just like a big RAG retriever.

Step 2: Create an Assistant object. To use an Assistant and conduct tasks, first, create an AI Assistant object. Supply the Assistant with a model, instructional behavior, tools to use, and file IDs to employ for its knowledge base, as parameters.

An Assistant represents an entity that can be configured to respond to a user's messages using several parameters like model, instructions, and tools.

API documentation is a collection of references, tutorials, documents, or videos that help developers use your API governed by the Open API Specification(OAS). An API(Application programming interface) is a data-sharing technique that helps applications communicate with each other. Not the best definition in the world but I like to think of an API as a dynamic messenger. They can store your message, process it, and also deliver it to multiple people. They are also responsible for the security of your message until it reaches you.

There are a lot of tools in the market used to produce great documentation; Swagger, Postman, Doxygen, ApiDoc, and Document360 just to name a few. However, most developers remain oblivious to the tools developed for reconnaissance which when you interact with them are useful to developers as well.

mitmproxy2swagger

mitmweb is a component of the mitmproxy project and it will serve to intercept the requests that will be channeled to the listener port opened at 8080

Next, we'll need to configure the requests source for which we'll use Postman

Next, click on the gear icon at the top right corner of the postman interface to access the settings

On the settings pop up select proxy and then toggle use custom proxy configuration Here we'll add the proxy listener port so that Postman can channel all request through out custom proxy from mitmproxy

inurl:”/api/v1" site:microsoft.com

Many times when the we are trying to Pentest an API we might not get access to Swagger file or the documentations of the API, Today we will try to create the swagger file using Mitmweb and Postman.

Man in The Midlle Proxy (MITMweb)

run mitmweb through our command line in Kali

and as we can see it starts to listen on the port 8080 for http/https traffic, and we will make sure that its running by navigating to the above address which is the localhost at port 8081

and then we will proxy our traffic thorugh Burp Suite proxy port 8080 because we already has mitmweb listening for this port (make sure Burp is closed)

and then we will stop the capture and use mitmproxy2swagger to analyse it