Analytical – Honest – Passionate
Some CSV files have issues with them. Below is a code snippet to save the information in a csv file into a duckdb database, ignoring all the ‘broken’ rows.
A code example for how to write BigQuery SQL to export a table to Google Cloud Storage in csv and parquet file formats. Parquet CSV
Data Ops, AKA Dev Ops for data. Essential content for learning about improving the velocity of development and effective maintenance of data products (products produced by data teams). Christopher Bergh …
Pandas is the go to library in Python for analysing and manipulating data. This post lists some useful video resources for learning this package. Brandon Rhodes – Pandas From The …
SQL is a great language for analysing and slicing and dicing data. The folks at Google who build Google Sheets appreciate this too. So much so that they have built …
An example of how to run multiple queries in parallel.
With the bq command line tool moving multiple tables at a time from one BigQuery dataset to another can be done with a few lines of code. The tool can …
A lightweight example of saving some logs to an SQLite database in a python script.
Use the Google technology stack to automatically load csv files received over email into BigQuery. In the data world, sending data over email is something that has been around for …
A short walkthrough guide of setting up a scheduled query in BigQuery. The Google documentation for scheduled queries is here. Step 1: Create a query in the bigQuery interface that …