Category "google-bigquery"

Bigquery: Match IP address to IP CIDR in another table

I am referring to this article in order to lookup IP addresses to country: https://cloud.google.com/blog/products/data-analytics/geolocation-with-bigquery-de-id

Teammate cannot edit bigquery scheduled query

I created a scheduled query in bigquery and I want my teammate to be able to edit it. However when they attempt to update and save the query there is a error me

How do I update a BigQuery table with multiple auto incrementing values?

I have managed to generate a single column based on multiple counts with different prefixes, as detailed here. But now I want to update an existing BigQuery col

program to Read content from 1 file and write string which is started from specific word and end with specific word into another file in python

Write a program to Read content from one file and write a string which is started from a specific word and ends with a different specific word into another file

How to cleanup BigQuery export to CSV using pandas

I exported my BigQuery data to CSV but can't figure out how to clean up the data as the headers are all appended on the backend in the same row. Here's my code:

How can I parse out of the last instance of text after a delimiter (big query)

Ex of value: [{‘test’:’https://‘,’id’,’2521’,’Shipment’}] I would only want to return Shipment as th

Running and joining two tables with a 100 million+ rows together

What is the best way to join two tables together which exceed rows of over 100 million rows? I'm running into an issue where my query is timing out after 6 hour

Repeated "Not found: Job" error message on tab opening

I'm getting the following error message every time I open any new tab in BigQuery, or when I move between tabs. It's been happening now for a couple of months a

How to consecutively count everything greater than or equal to itself in SQL?

Let's say if I have a table that contains Equipment IDs of equipments for each Equipment Type and Equipment Age, how can I do a Count Distinct of Equipment IDs

How to insert data from CTE to a Temp Table?

I am trying to create a some logic using CTE and then instead of using DML statement after CTE, I am trying to create a temp table using CTE. This is possible i

selecting columns which are NOT of a specific type SQL

I want to select all the columns in BigQuery that are not of the type "TIMESTAMP". I have written the query which returns such columns which is: SELECT COLUMN_N

Querying from BigQuery to Python

I'm trying to query data to bigquery with this code as mentioned in google docs: from google.cloud import bigquery client = bigquery.Client() tab = client.query

BigQuery DENSE_RANK

I posted something similar to this but without the full code and I don't think people understood what I was looking for... I'm writing a query to examine bus st

Get Syntax Error: Expected end of input but got keyword ORDER at [5:1]: in BigQuery

When I run the following I get the error Syntax Error: Expected end of input but got keyword ORDER at [5:1]: SELECT purchase_price FROM test-project-349410.cust

Apache Beam with Dataflow: flag 'ignore_unknown_columns' for WriteToBigQuery not working

I am building a streaming pipeline using Apache Beam (Python SDK version 2.37.0) and Google Dataflow to write some data I receive via Pubsub to BigQuery. I proc

SQL: how to check if at least one keyword from a keyword list exists in a string

I have a table with the following schema, id data_string 1 I have a pen. 2 Monday is not good. 3 I love Friday. ... ... And I also have a list of k

Count overnight hours as one day

I have a dataset where certain operations occur during the overnight hours which I'd like to attribute to the day before. For example, anything happening betwee

What is the formula in SQL to calculate time/ number of days taken to sell x units of a product since its launch?

As part of a dashboard development, I need to know the number of days taken by a newly launched item to reach sales of 1000 units. This is for benchmarking perf

Querying JSON value in BigQuery

I have a BigQuery table ("activity") where one column ("groups") contains a JSON value (the table is an export of Google Workspace logs). I'd like to be able to

SCD2 on nested table in GCP big query

I am new to gcp big query. I am trying to model scd2 implementation on both outer and nested table on GCP big query. I would like to know your suggestions wethe