Memory Use

When working with data, especially relatively large tables, it is often important to keep an eye on the table’s size and at times take actions to optimize the query to avoid scanning unnecessary data, change the data types of certain column, or work with a smaller subset of the data.

In this page, we look at how to get information about the size of a table.

Table

We wish to know how much memory a particular table consume.

BigQuery Console

The best option is to use the BigQuery console to get all the storage information. Details tab provides info about both physical and logical storage.

inspecting-details

Metadata

We wish to get the storage consumption of a table using a SQL query.

SELECT size_bytes, ROUND(size_bytes / POW(10, 9), 2) AS size_gb
FROM refcon.dataset.__TABLES__
WHERE table_id = 'table_1';

Here is how this works:

  • __TABLES__ is a hidden metadata table that exists inside each BQ dataset.
  • We filter the results using the target table name to get the size in bytes.
  • To convert the GB we divide by $10^9$.

Particular Column

We wish to know how much memory a particular column consumes. The best option is to use BigQuery console as it shows us how much data a column consumes in the top right corner by selecting only that column.

SELECT col_1
FROM refcon.dataset.table_1;

inspecting-column-size

SQL
I/O