This document describes the AWS Well-Architected Data Analytics Lens, a collection of customer-proven best practices for designing well-architected analytics workloads. The Data Analytics Lens contains insights that AWS has gathered from real-world case studies, and helps you learn the key design elements of well-architected analytics workloads along with recommendations for improvement. The document is intended for IT architects, developers, and team members who build and operate analytics systems.
Build an automatic data profiling and reporting solution with Amazon EMR, AWS Glue, and Amazon QuickSight
This post demonstrates how to extend the metadata contained in the Data Catalog with profiling information calculated with an Apache Spark application based on the Amazon Deequ library running on an EMR cluster. You can query the Data Catalog using the AWS CLI. You can also build a reporting system with Athena and Amazon QuickSight to query and visualize the data stored in Amazon S3.
Packaging a python library - Thoughts on packaging python libraries
Thoughts on packaging python libraries — Note This is about packaging libraries, not applications. ⸻ All the advice here is implemented in a project template (with full support for C...
Devart Excel Add-ins to Connect with Cloud Apps and Databases
Devart Excel Add-in for PostgreSQL allows connecting Microsoft Excel to PostgreSQL data, import data from Excel to PostgreSQL, edit these data and save them back to PostgreSQL.
If the original data is spread out in a number of files, we can read them in parallel (on multi-core) machines to save time. Ideally. However, it seems a bit tricky with python.
This thoroughly revised guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small yet powerful command-line tools to quickly obtain, scrub, explore, and model your data. To get you started, author Jeroen Janssens provides a Docker image packed with over 100 Unix power tools—useful whether you work with Windows, macOS, or Linux.
Become a Data Engineer on a Shoestring (aka The Best Free Courses and Learning Resources) - Data Engineering Coding Bootcamp — Pipeline Data Engineering Academy
I was tinkering with the idea of finding the right way to help others to identify the resources that give you bang for the buck when it comes to upskilling yourself in data engineering… and this is what I came up with. The best data engineering courses, learning materials, data engineering training.
Migrate SQL Server to Amazon Aurora PostgreSQL using best practices and lessons learned from the field | Amazon Web Services
Many organizations are benefiting from the performance gains and ease of administration of the Amazon Aurora database platform. The idea of breaking free from commercial licenses, backup administration, and data center maintenance is always a welcome thought. But where to begin, and how complex is the journey to migrate from SQL Server to Amazon Aurora […]
How to write effective Stored Procedures in PostgreSQL | Packt Hub
In this article, we explain the concept of Stored Procedures, and how to write them effectively in PostgreSQL 9.6. PostgreSQL is an open source database
Business logic is supposed to be the part of the application where you
deal with customer or user facing decisions and computations. It is often
argued that this part should be well separated from the rest of the
technical infrastructure of your code. Of course, SQL and relational
database design is meant to support your business cases (or user stories),
so then we can ask ourselves if SQL should be part of your business logic
implementation. Or actually, how much of your business logic should be SQL?
Part I. Tutorial Welcome to the PostgreSQL Tutorial. The following few chapters are intended to give a simple introduction to PostgreSQL, relational …
We here are very big fans of Postgres as a database and believe it is often the best database for the job. For many though, working with and maintaining Postgres involves a steep learning curve. This guide is designed as an aid for beginners and experienced users to find specific tips and explore tools available within Postgres.
If you want to take the quick & painless approach instead, there are great services available for Postgres hosting.