Development

Development

2136 bookmarks
Newest
Soil health data cube - AI4SoilHealth
Soil health data cube - AI4SoilHealth
AI4SoilHealth is building a Soil Health Data Cube to aid the sustainable management of Europe’s soils by integrating crucial data on soil, climate, and
·ai4soilhealth.eu·
Soil health data cube - AI4SoilHealth
How to Generate JSON Schema Effectively and Efficiently
How to Generate JSON Schema Effectively and Efficiently
Creating a JSON schema can be a crucial step in ensuring data consistency and quality, especially when dealing with APIs and data exchange. Below is a detailed guide on how to create a JSON schema:
·apidog.com·
How to Generate JSON Schema Effectively and Efficiently
Getting Creative with OpenAPI - by David Biesack
Getting Creative with OpenAPI - by David Biesack
Defining resource creation and update operations with OpenAPI
unevaluatedProperties: false
This article pulls in lots of aggregated knowledge from earlier articles in the Language of API Design series: Mapping a domain model to OpenAPI Patterns for clean URLs and URL structure Defining the request body for a POST operation Defining API responses with OpenAPI response objects and response body data format with JSON Schema Composing JSON Schemas to keep the OpenAPI definition DRY Understanding the subtleties of JSON schema (notably, unevaluatedProperties) Defining reusable response objects to keep the OpenAPI definition DRY Defining how the API handles problems with client requests as well as server errors
·apidesignmatters.substack.com·
Getting Creative with OpenAPI - by David Biesack
Validating API Requests - by David Biesack
Validating API Requests - by David Biesack
Techniques for API Request Validation
A promise of REST APIs is good decoupling of clients and services. This is achieved in part by reducing business logic as much as possible in the client application. For example, a client application may use a form for collecting information used in a POST operation to create an API resource, or to edit an existing resource before updating it with a PUT or PATCH operations. The client then maps the form fields to the properties of the operation’s request body. Clients can use front end frameworks and libraries to perform lots of low-level validation in the front end corresponding to JSON schema constraints.
However, this only covers “syntactic” or static field-level validation. Often, an API will also have business rules that the client must follow. Secure API services will enforce those business rules in the API operations
Parse the options and (JSON) request body and return a 400 Bad Request if any of the request data is malformed (i.e. does not satisfy the constraints of the operation (such as required body or required parameters) or all the JSON Schemas associated with the operation’s parameters or request body) Verify that the caller passes valid Authorization to the API call, and return 401 Unauthorized if not Verify that the caller is authorized to perform the API operation, and return a 403 Forbidden error if not. Verify the state of the application and return 409 Conflict if the operation would put the application into an inconsistent state Verify the semantics of the request body and return a 422 Unprocessable Content error if the request is incomplete, inconsistent, or otherwise invalid
One pattern is to extend the API operations with a dry run feature. A dry run is a variant of the API operation which performs all (and only) the validation performed by the full operation, but does not execute the associated behavior. As such, it will return the same 400/401/403/409/422 that the full operation would return, allowing the client to highlight invalid form data or otherwise correct the problem. The client can use dry run operation incrementally as the user fills out a form, and disable the “Save” or “Submit” or similar UI controls if there are validation errors.
One way to implement a dry run is to create a separate "validation” operation for each API operation. This has the significant disadvantage of greatly increasing the footprint (size) of the API and adding a lot of duplication.
Rather than duplicate operations to add sibling validation operations, another approach is to add a ?dryRun=true query parameter to the operations. When used, the operation can return 204 No Content if the request contains no problems. The dryRun parameter acts as a “short circuit” in the API operation. The implementation performs the full validation it would normally do before executing the desired behavior, but then stops before actually executing anything other than the validation.
This pattern has a small impact on the API footprint compared to making sibling validation operations. A smaller footprint makes the API easier to read and understand. It is also a good use of the DRY principal, since you do not have to duplicate the definition of all the operation request parameters and request bodies, which opens up the chance for them to become out of sync.
·apidesignmatters.substack.com·
Validating API Requests - by David Biesack
Leave a calling card - by David Biesack
Leave a calling card - by David Biesack
Make API responses more self-descriptive with reference objects
In OAS, a reference object contains a $ref value and may contain summary or description values. However, there is a kernel of usefulness here that can be used outside of defining an API with OpenAPI — we can use a construct inspired by OAS reference objects instead of terse and cryptic Restful JSON URLs or naked resource ID properties.
Such reference objects in API responses (or requests) can include other key identifying data about the referenced resource; these are optional and informative. The id or url of the resource are required to actually identify the referenced resource. This makes the overall JSON payload more self-descriptive and does not send the developer down ratholes trying to understand data they see when exploring and learning the API.
To make these reference objects even more useful, the API service can build them for you instead of making the client construct them. For example, the response from the getUniverse operation can contain a reference or ref property which is the reference object that the client can embed in other requests when citing a universe. The *Item schema in the API’s list responses (see “What am I getting out of this?”) can also include the reference object.
·apidesignmatters.substack.com·
Leave a calling card - by David Biesack
Bump CLI
Bump CLI
This site contains the Bump.sh documentation, product updates descriptions, examples of how to use Bump.sh, as well as guides to help you build better REST APIs (OpenAPI) and event-driven architectures (AsyncAPI).
·docs.bump.sh·
Bump CLI
Real Estate API documentation
Real Estate API documentation
Real Estate API - Focused Specification > OpenAPI specification for tailored for use with the {landrise.reapi} R package. Endpoints Includes specifications for the Real Esta...
·bump.sh·
Real Estate API documentation
Bump.sh Documentation
Bump.sh Documentation
This site contains the Bump.sh documentation, product updates descriptions, examples of how to use Bump.sh, as well as guides to help you build better REST APIs (OpenAPI) and event-driven architectures (AsyncAPI).
·docs.bump.sh·
Bump.sh Documentation
Composing API Models with JSON Schema
Composing API Models with JSON Schema
Use JSON Schema effectively to build real API request and response bodies
This approach of defining tiny schemas, such as the chainLinkContentField, is a useful design pattern for keeping API definitions DRY. This is analogous to creating small, atomic, highly reusable functions in a programming language, rather than long, complex single-purpose functions. If you find yourself repeating a property definition across multiple schemas, consider lifting that property into its own field definition schema. If there are groups of related properties that you use together in multiple schemas, you can group them together into a reusable {group}Fields schema (with a descriptive schema name that indicates its purpose, such as mutableChainLinkFields), then mix them into other schemas using the allOf schema composition construct. This practice is one form of refactoring that is useful when defining and refining schemas.
Composition is not Inheritance
Warning: this JSON Schema composition construct is not inheritance… even if some tools translate this into inheritance in a target language. This means that, as the “allOf” name implies, all the schema constraints apply: you cannot override or replace constraints.
authorId: description: >- The ID of the author resource: who created this chain link. type: string minLength: 4 maxLength: 48 pattern: ^[-_a-zA-Z0-9:+$]{4,48}$ readOnly: true
Suppose we want to use schema composition but loosen those constraints in the chainLink schema’s authorId to allow 2 to 64 characters, we can express this as follows:
chainLink: title: A Chain Link allOf: - $ref: '#/components/schemas/chainLinkItem' - type: object properties: authorId: description: >- The ID of the author resource: who created this chain link. type: string minLength: 2 maxLength: 64 pattern: ^[-_a-zA-Z0-9:+$]{2,64}$ readOnly: false
However, we won’t get the desired effect. While the code generation may not change, the schema validation of a JSON object containing an authorId still enforces all of the subschema constraints. Thus, the effective schema constraints for the authorId property are the union of all the constraints: type: string minLength: 4 maxLength: 48 pattern: ^[-_a-zA-Z0-9:+$]{4,48}$ type: string minLength: 2 maxLength: 64 pattern: ^[-_a-zA-Z0-9:+$]{2,64}$
While a value such as “C289D6F7-6B30-4788-9C70-4274730FAFCA” satisfies all 8 of these constraints, a shorter author ID string of 3 characters, “A00”, will fail the constraint #2 and #4, and thus that JSON would be rejected.
A final note on schema composition. The behavior of the allOf construct is well defined within JSON Schema for JSON validation. However, there is no standard for how this JSON schema construct must be treated in other tools such as code generation for client or server SDKs. Thus, each tool vendor (SmartBear’s swagger-codegen, the OSS openapi-generator, ApiMatic, Speakeasy, Fern, Kiota, etc.) may interpret JSON Schema in individual and different ways. It is useful to try them out to see if their interpretation and implementation meets your needs and expectations.
JSON Schema provides other keywords for schema composition (the oneOf, anyOf, and not keywords, as well as conditionals with a if/then/else construct) but these are more advanced topics and I’ve already asked too much of you to read this far. Stay tuned, however, there is more to come!
·apidesignmatters.substack.com·
Composing API Models with JSON Schema
Validating API Requests
Validating API Requests
Techniques for API Request Validation
A promise of REST APIs is good decoupling of clients and services. This is achieved in part by reducing business logic as much as possible in the client application. For example, a client application may use a form for collecting information used in a POST operation to create an API resource, or to edit an existing resource before updating it with a PUT or PATCH operations. The client then maps the form fields to the properties of the operation’s request body. Clients can use front end frameworks and libraries to perform lots of low-level validation in the front end corresponding to JSON schema constraints
Forms which use required data fields for properties that are required in the JSON schema using date pickers checkboxes for selecting Boolean true or false values drop down lists that allows selection from a list of fixed enum values constrained numeric text entry form fields that enforce a regular expression from a pattern constraint
However, this only covers “syntactic” or static field-level validation. Often, an API will also have business rules that the client must follow. Secure API services will enforce those business rules in the API operations
Parse the options and (JSON) request body and return a 400 Bad Request if any of the request data is malformed (i.e. does not satisfy the constraints of the operation (such as required body or required parameters) or all the JSON Schemas associated with the operation’s parameters or request body) Verify that the caller passes valid Authorization to the API call, and return 401 Unauthorized if not Verify that the caller is authorized to perform the API operation, and return a 403 Forbidden error if not. Verify the state of the application and return 409 Conflict if the operation would put the application into an inconsistent state Verify the semantics of the request body and return a 422 Unprocessable Content error if the request is incomplete, inconsistent, or otherwise invalid
One way to implement a dry run is to create a separate "validation” operation for each API operation. This has the significant disadvantage of greatly increasing the footprint (size) of the API and adding a lot of duplication.
Rather than duplicate operations to add sibling validation operations, another approach is to add a ?dryRun=true query parameter to the operations. When used, the operation can return 204 No Content if the request contains no problems. The dryRun parameter acts as a “short circuit” in the API operation. The implementation performs the full validation it would normally do before executing the desired behavior, but then stops before actually executing anything other than the validation.
This pattern has a small impact on the API footprint compared to making sibling validation operations. A smaller footprint makes the API easier to read and understand. It is also a good use of the DRY principal, since you do not have to duplicate the definition of all the operation request parameters and request bodies, which opens up the chance for them to become out of sync.
·substack.com·
Validating API Requests
Learn - OpenAPI Spec
Learn - OpenAPI Spec
OpenAPI helps speed up API development. You Define, mock, and test REST APIs using a single truth/specification. Ideal for dev and QA teams adopting contract-first workflows.
Deeply nested schemas can become unwieldy and hard to maintain. For instance, a User object containing an Address object, which in turn contains a Location object, can quickly become complex. Why it matters: Simplifying schemas enhances readability and maintainability, making it easier for both developers and consumers to understand and work with the API.
Defining schemas, parameters, and responses inline repeatedly instead of using the components section leads to redundancy and potential inconsistencies. Why it matters: Leveraging components promotes reusability and consistency across the API specification.
Logically group your APIs into smaller, domain-specific specs — like auth.yaml, payment.yaml, orders.yaml. Use tags in OpenAPI to group related endpoints (like Order, Customer, Admin) even within a single file if needed.
/openapi ├── auth.yaml ├── customer.yaml ├── orders.yaml └── components/ └── common-schemas.yaml
·beeceptor.com·
Learn - OpenAPI Spec
Best API Documentation Tools | Beeceptor
Best API Documentation Tools | Beeceptor
Find the detailed review with tabular comparison of modern API documentation tools for creating developer friendly API documentation, helping developers integrate APIs seamlessly and efficiently.
Start with Real-World Examples: Show complete request/response cycles, including auth headers and errors. For example, show what a GET /users call returns in JSON.
Modern Tools​ Redocly: Offers Redoc for API docs plus additional tools like Revel for flexible branding without rigid templates, Reef for API monitoring, and an API registry to manage multiple OpenAPI definitions. Theneo: Uses AI to generate API references automatically, cutting down manual work. Started with docs but now expands its AI tools to streamline the whole API development process. Stoplight: Helps with API design and governance through tools like Stoplight Studio (visual OpenAPI editor), Prism (open-source HTTP mock server), and Spectral (linting tool to enforce API standards). Works best for teams taking a design-first approach. Postman: Goes beyond basic docs by providing workspaces, automated testing, monitoring, and governance. Connects deeply with GitHub, GitLab, and CI/CD pipelines as a complete API lifecycle tool. SwaggerHub: Lets teams collaborate on APIs using OpenAPI or AsyncAPI specs while managing them throughout their lifecycle. Built by the same team behind Swagger. Zuplo: Runs as a lightweight, fully-managed API platform built for developers, featuring GitOps, quick deployment, and unlimited preview environments. Works for both hobbyists and engineering leaders implementing auth systems. Gravitee.io: Works as an open-source API management platform with a full ecosystem including API Gateway, Management, Access Management, and observability tools. APIdog: Simplifies creating, managing, and testing APIs for both developers and testers. Focuses on building reliable, secure, and fast APIs. ReadMe: Helps companies create, manage, and publish API docs through a user-friendly interface that makes documentation accessible to users. dapperdox.io: Generates and serves open-source API docs for OpenAPI Swagger specs. Combines specs with docs, guides, and diagrams using GitHub Flavored Markdown. Docusaurus: Built by Meta as an open-source documentation platform. Lets developers write in Markdown or MDX and, being React-based, allows extensive customization. Scalar: Turns OpenAPI specs into interactive docs with an integrated API playground for testing endpoints directly in the documentation.
·beeceptor.com·
Best API Documentation Tools | Beeceptor
API Errors | Beeceptor
API Errors | Beeceptor
This guide to help build great API experience for the consumers. Discover top errors, their fixes, from authorization to data handling, and enhance the developer experience!
Example of an Actionable API Error Message​ { "status_code": 400, "error": "Bad Request", "message": "The 'email' field is missing or invalid.", "suggestion": "Please provide a valid email address in the format 'user@example.com'.", "error_code": "VALIDATION_FAILED_001", "trace_id": "xyz1234abcd" } This message clearly identifies: The nature of the error (400 Bad Request) The specific problem (missing or invalid email field) A suggested fix (provide a valid email in the correct format) An internal error code for reference A trace ID for further investigation
·beeceptor.com·
API Errors | Beeceptor
401 Unauthorized vs 403 Forbidden
401 Unauthorized vs 403 Forbidden
Find the key differences between HTTP status codes 401 Unauthorized and 403 Forbidden with tabular comparison including when to use each in API development, with practical examples.
401 Unauthorized vs 403 Forbidden
In web development, ensuring access control is essential in safely and efficiently managing APIs. The meanings of 401 Unauthorized and 403 Forbidden are sometimes confused. Nonetheless, both codes have to do with restricted resources, but they serve different purposes. In this article, we will explain the codes and instruct you on which one to use.
401 Unauthorized?​ The response is an HTTP error code for a request lacking valid authentication credentials from a client is referred to as the 401 Unauthorized status code. That being said, it means that before accessing the requested resource, it’s necessary for the server to authenticate itself to the client. If no credentials are provided or if wrong ones are given by the client, then what follows is a 401 status code.
When to Use 401 Unauthorized​ Use 401 Unauthorized when: No authentication details have been received yet from the client. The authentication information supplied – username and password/token – is not valid/has expired. There is no authorization header present in your requests like “Authorization.” For instance, if an API demands Bearer token for access but this token has not been included in any request or is incorrect it will issue back a response having HTTP status code 401 Unauthorized (the most common case).
403 Forbidden?​ The reason for using a 403 Forbidden status code is when the server recognizes the request, the client has been authenticated, but the client does not have permission to access the requested resource. It means that in this case, a client is known while a server intentionally turns down fulfilling the request because of inadequate privileges. When to Use 403 Forbidden​ Use 403 Forbidden when: Authenticated clientele lack sufficient permissions to reach given resources. Server denies resource access irrespective of client’s authentication state. Client’s access to resources is prohibited by any form of an access control system. For instance, an authorized user may try accessing an admin only page without having adequate role. Even if one gets logged in, the response will indicate 403 Forbidden if they do not have sufficient rights.
·beeceptor.com·
401 Unauthorized vs 403 Forbidden
How to Easily Capture and Test Code Output in R
How to Easily Capture and Test Code Output in R
Learn methods to capture and test code output in R, including snapshot testing, dput, and constructive package.
·jakubsobolewski.com·
How to Easily Capture and Test Code Output in R
Databot is not a flotation device - Posit
Databot is not a flotation device - Posit
Databot is an exciting new LLM tool for exploratory data analysis, but to use it safely and effectively, you still need the critical skills of a data scientist.
·posit.co·
Databot is not a flotation device - Posit
Lightweight Object-Relational Mapper for R
Lightweight Object-Relational Mapper for R
oRm is a lightweight Object-Relational Mapper (ORM) for R. It simplifies database interactions by allowing users to define table models, insert and query records, and establish relationships between models without writing raw SQL. oRm uses a combination of DBI, dbplyr, and R6 to provide compatibility with most database dialects.
·kent-orr.github.io·
Lightweight Object-Relational Mapper for R