AWS Glue Databrew: A Guide to Visual Data Preparation

Raw, Unstructured Data

This can include system logs, sensor data, text files, and so forth. Raw data is generated as-is, without further processing. It has no model or structure. Raw data is suited to platforms like AWS data lakes, which can house and load large unstructured data sets.

Semi-Structured Data

An application or service may output data with a basic model or structure. This includes Excel files, which use the comma-separated values (CSV) and columns or rows to form a hierarchy.

Structured Data

Structured data could include a combined list of first and last names, dates, zip codes, or payment card information. Enterprises can query structured data using SQL, NoSQL, or any other data language.

Visualized Data Preparation

Where most data are viewed as alphanumeric values in columnar databases, DataBrew works different. It visualizes all loaded data sources so you can understand the hierarchy and data relations. On the topic of no code, the graphical user interface (GUI) allows users to click, drag, and edit data without a single line of code

250+ Data Preparation Automations

Data scientists follow several isolated, repeatable workflows and processes as part of their role. AWS has modeled these processes and workflows as language and data-agnostic modules, creating a library of actions that are usable by end users. This can include creating pivot tables, handling missing data values, data filters, and data encoding. Then, you can create a DataBrew recipe to create singular or batch action automation templates.

Data Lineage

Much like audit logs are used to track customer activities in an IT network, data lineage allows you to track data transformation activities within AWS Glue DataBrew. This includes the data source, any transformations that have been applied, and the data output including target location. With transformations, it covers how data was transformed, what changed, and why it was transformed, so that everyone can interpret the chain of events. Much like a family tree, this allows you to go back and understand how the data came to be.

Data Mapping

Trying to find the similarities between two databases? Databrew helps you find matching fields that are present in two data sources. As matching fields are identified, they are loaded into a schema. This enables the creation of a homogenous, centralized database that condenses multiple data sources into one.

Reduced Barrier of Entry to Data Preparation

The no-code format of the platform means anyone, of any skillset, can prepare data. While functionality is not advanced enough to replace data engineers, it allows non-technical employees to perform an ETL process, data exploration, and data quality workflows much like data analysts would. This reduces the barrier of entry to data preparation, which directly tackles the shortage of data science personnel across all industries.

Automated Data Profile Creation

As you work on projects in DataBrew, you will be shown a dataset preview, profile overview, column or row statistics, and a full data lineage. This falls under a data profile, giving you a snapshot overview of the new dataset.

Automate 250+ Common Data Preparation Processes

Using models for data preparation can significantly reduce the amount of effort required. Rather than writing a line of code, or copying a code template and editing the parameters, you can simply configure and execute an action to prepare data. This is known as workflow enhancement, speeding up low-level repeatable processes to get more done in less time.

Intelligent Prescriptive Suggestions

Glue DataBrew incorporates light prescriptive analytics while you work. This means you get suggestions to speed up preprocessing workflows, reducing utilization costs with a faster time to benefit (FTTB).

No Code Means No Code

DataBrew uses standardized, tested modules that write the code for you while allowing editing of parameters through a GUI. When AWS says “no code,” it really does mean no code. However, you cannot introduce custom code to this platform, as it is heavily locked down to the 250+ data transformation actions. For advanced technical users, this may be a deal breaker. But most non-technical users will appreciate the no-code nature of DataBrew.

Cloud Only Leads to No Offline Support

Since AWS is a cloud service provider, naturally there is no offline support. It is web only, accessible through a web application. This means that on-premises data centers cannot use their LAN datasets in DataBrew if broadband connections fail — a complication for hybrid cloud users.

No Custom Visualizations

The no-code nature of the platform means visual interface customizations of any sort are scarce. You are limited to DataBrew visualizations, such as basic graphs, charts, and bars, rather than integrating third-party services like PowerBI, Tableau, or Looker.

Per Session Pricing

DataBrew sessions are charged on a per-session basis. Each session costs $1.

Session Timescales

AWS DataBrew sessions are 30 minutes each. Each session is interactive, so moving the mouse means the clock is ticking. No activity at the end of a 30-minute period

Per Node Pricing

DataBrew jobs use nodes to calculate pricing. Each node includes four virtual CPUs and 16GB of RAM. The default for each job is five nodes.

Node Hours

A node hour costs $0.48. This is calculated per minute.

Additional AWS Services

You will be charged for each additional service if you use supplementary AWS services alongside DataBrew jobs. For example, the AWS Glue Data Catalog.

Pandas

Pandas is an open-source tool for data analysis and manipulation. Built in the Python programming language, it allows users to create objects, view and select data, populate missing data, merge datasets, and much more. It supports CSV, TXT, HTML, JSON, LaTeX, XML, SQL, SAS, and SPSS as “IO tools.”

NumPy

Numerical Python (NumPy) is another tool used across all data industries. It’s an open-source Python library focused on numerical data (Pandas is actually based on this as a source). It allows users to create arrays, add and remove elements, alter shapes and sizes of arrays, convert from 1D to 2D arrays, and create or reshape matrices.

Alteryx

Where the previous two competitors are free and open source, Alteryx is a paid automated analytics service. It matches DataBrew with data quality and preparation tools, including data access, catalogs, exploration and profiling, and enrichment. More broadly, Alteryx also offers an analytics cloud, analytics process automation, and machine learning.

Dask

Dask is another free, open-source Python library. It is built in coordination with NumPy and Pandas. It allows users to create a Dask array, Dask bags, DataFrame, collections, and more. The focus is enabling parallel, scalable Python solutions for data preparation using schedulers that execute task graphs.