Questions tagged [google-cloud-platform]
Google Cloud Platform provides cloud computing services that let you build applications and websites, store data, and analyze data on Google's infrastructure.
48,508
questions
0
votes
0
answers
9
views
Can Google Identity Platform send claims tokens back to the server?
I'd like to integrate Identity Platform with Apache's mod_auth_openidc to provide SSO on a site. However, I'm having trouble understanding how to do this, or even if it's possible.
I can get basic '...
0
votes
0
answers
5
views
Execute DDL to create a native table in BigQuery from files stored on GCS
I would like to create a native table in BigQuery from a CSV file with DDL statements in a way that replicates the "create table from..." feature in BigQuery's UI as shown in the screenshot ...
0
votes
0
answers
6
views
Does domain migration result in data loss in zoho email service provider?
I purchased my domain in AWS.
And then I set my txt and mx records from zoho email service provider in AWS route 53 and activated my custom domain email in zoho.
Now I want to migrate my domain from ...
0
votes
0
answers
5
views
specify task_type for embeddings in Vertex AI
Has someone tried the last update of GCP TextEmbeddingInput that allows to specify the task_type of your application? Theoretically it should allows you to use different fine tuned models to generate ...
0
votes
0
answers
13
views
Firebase Hosting CDN - Sometimes miss already cached images
We use the CDN to cache transformed images from a media server, but occasionally while using our website we see a slow downloading images, witch should be already cached. I can see that the image is ...
0
votes
2
answers
21
views
Can't add scripts on VM itself as pre/post patch scripts in GCE VM Manager Patch
On the VM I am trying to patch, I have two scripts in my main user's home directory: /home/myuser/pre-patch.sh and /home/myuser/post-patch.sh. Both have been chmod +x'ed.
I am trying to set these ...
0
votes
0
answers
9
views
configuring dataproc with an external hive metastore
I'm trying to create a cluster on GCP Dataproc with an external Hive metastore.
I need that hive metastore writes data in GCS.
So i started to create a Mysql :
gcloud sql instances create hive-...
0
votes
0
answers
17
views
How to execute SQL from Dataflow Streaming job in a periodic manner with checking the state of the initial query?
I need to create a dataflow streaming pipeline in GCP with these requirements.
Read data from Kafka and do streaming insert to a table say table1 in bigquery.
Do an SQL operation in bigquery from the ...
-1
votes
0
answers
16
views
Error invoking callback , String length JS RangeError: String length exceeds limit
getting string length range error while converting a 300MB size video to arraybuffer using fetch.
fetch(path)
.then(async (value) => {
value
.arrayBuffer()
.then((...
0
votes
1
answer
21
views
Google Cloud IAM: website hosted in Cloud Storage to trigger Cloud Function (Gen 2)
I'm working inside Google Cloud, completely inside single project.
My current issue is associated with corrected access rights only.
I've developed a simple website hosted on Cloud Storage and having ...
0
votes
0
answers
19
views
How to resolve pipy SSL verification failure
Dockerfile
RUN set -ex && pipenv sync
Pipfile
[[source]]
name = "pypi"
url = "https://pypi.python.org/simple"
verify_ssl = true
Error message when I tried to build and ...
0
votes
0
answers
6
views
What does "in use by" mean for Google Cloud VM's
When I look at VM instances in my Google Cloud console I see a column "in use by"
I see two VM instances (e,g vm-a and vm-b) "in use by" vm-c . I also see an instance template ...
0
votes
1
answer
25
views
Firestore Document Maps to List of Objects Kotlin
I have this Firestore Document:
Every field in the document is a Map. I would like to obtain these fields as a List<Schedule> in Kotlin but I couldn't get it.
I have this:
data class Schedule (
...
0
votes
1
answer
69
views
Manage access to GCS buckets using VPC Service Controls
I have three GCS buckets (A, B, C) in one project and these buckets are controlled by VPC Service Controls - only one specific IP address range can access these buckets.
Is it possible to make one of ...
0
votes
0
answers
16
views
How do I find my Google Adsense API "parent" account?
I am trying to set up the Google Adsense API for a project I am working on and have started playing around with the Google OAuth 2.0 playground. Some of the commands in the list they provide reference ...
0
votes
0
answers
29
views
Google VM connecting Global ELB
One of my app needs to connect to two services (based on the URL path having same base path) in some other VPC (and in few cases in other region as well). Both apps are on Google VPC.
My requirement ...
-2
votes
0
answers
15
views
Is Windows Per Core Licensing Costs the only option Windows Server 2022? [closed]
Is there a way to reduce the Windows Licensing cost on GCP?
I'm not a Licensing expert, but it appears that licensing itself costs more than the actual compute. I can't imagine this is what Microsoft ...
0
votes
1
answer
15
views
Create GCP IAM to subscribe and also list only a dedicated subscription
I want to achieve that an extra created service-account is only possible to subscribe to a certain pubsub-subscription.
And also, when one uses this service-account for listing all subscriptions, it ...
0
votes
1
answer
19
views
How do I iterate through JSON files stored in GCP bucket in different folders. Example; | Bucket/Dict/Folder2/file.json Bucket/Dict/Folder1/file.json
I have dumped JSON files from DOCAI to GCP but each file is stored in individual folder, although they are in the same bucket on Cloud Storage. I am not able to iterate through the JSON files stored ...
0
votes
1
answer
28
views
When creating a Cloud Data Loss Prevention (DLP) job where is the metadata later found?
Python create_dlp_job method
I want to associate some additional data with a DLP job.
The create_dlp_job method has a metadata parameter. Is there a method that allows me to retrieve that metadata ...
2
votes
1
answer
21
views
How to redirect traffic for 404 error to a different url when using a Cloud Run service to proxy a GCS bucket?
Currently I am using Cloud Run service to proxy a GCS bucket. Used the following https://github.com/domZippilli/gcs-proxy-cloud-run to use as a proxy. Now I want to redirect the traffic for 404(Not ...
0
votes
0
answers
14
views
bigquery operation time out in europe west 3
All of our datasets have location europe-west3 in BigQuery and it's been going fine until last weekend. Suddenly all read queries fail to retrieve any data and hit operation timeout. When we changed ...
0
votes
1
answer
43
views
Using conditional in Terraform and GCP
I'm working with a cloud sql module developed in my company. The module creates everything, this means the instance, database, Service Accounts, Firewalls, Private IP, Public IP, etc.
Our problem now ...
0
votes
1
answer
24
views
Is there a way to document AVRO schema for GCP pubsub
enter image description hereLike openAPI is there a way to document AVRO schema for GCP pubsub. Is there an UI provided by GCP like swagger UI. Or what is the recomendation here.
According to me Async ...
0
votes
1
answer
19
views
GCP Pubsub get the delivery attempts of a message on pull
I am receiving data from GCP pub/sub using a pull subscription.
I don't get any delivery_attempts field.
I saw it here: https://cloud.google.com/pubsub/docs/samples/pubsub-dead-letter-delivery-attempt
...
0
votes
0
answers
27
views
BigQuery SLA and service credits
Does anyone have experience measuring BigQuery's SLA and claiming service credits when it is out of SLA?
The SLA is defined by API error rate (see full details here), and GCP only provide credits if ...
0
votes
0
answers
22
views
Inquiry Regarding Unfamiliar Charges for "Dataplex" in GCP Billing
I am currently running queries for aggregations such as GA in GCP's BigQuery. The queries are straightforward, and the data volume is still relatively low, allowing me to perform aggregations within ...
0
votes
1
answer
13
views
Cloud Run/Build artifacts buckets are created with Fine Grained access policy by default
When using Cloud Run Jobs (I'd assume any service running Cloud Build under the hood as well), dedicated artifact buckets are created. Their naming is usually something like: us.artifacts.project....
0
votes
0
answers
52
views
GCP Cloud SQL and DLP (Data loss Prevention) [closed]
I want to implement DLP for my Cloud SQL postgres instance but I don't find any example or use case.
I found this repo that explain and shows how to detect sensitive data in cloud sql but It's ...
0
votes
0
answers
25
views
UnboundLocalError: cannot access local variable 'image_bytes' where it is not associated with a value
quite new to the programming world and trying to create a Streamlit web app for image classification (GCP AutoML, single-label model, hosted in a GS bucket).
I got the following error message; ...
0
votes
0
answers
24
views
Ruby on Rails behind nginx on Google Cloud Run fails to start
I'm having a weird problem deploying Ruby on Rails to Google Cloud Run. I made a main container (nginx) that works on the default Cloud Run PORT (8080) and 2 sidecars (rails on port 3000 and Next.js ...
0
votes
0
answers
7
views
Modify Cloud Trace Sampling Rate for Cloud Run
I am running NodeJS code in Cloud Run, via Firebase functions (v2). I am tracing my functions using OpenTelemetry and exporting to Cloud Trace.
I understand from the documentation here
Cloud Run ...
0
votes
0
answers
20
views
How do you peek gcp pubsub messages?
I have a pubsub topic that has subscription with a few unacked messages and I would like to peek at these messages. Is it possible?
The REST APIs doesnt seem to have a method that allows that. Should ...
0
votes
0
answers
17
views
How to configure my virtual machine on google cloud to use it with ssh on my normal terminal [closed]
I am seeking guidance on configuring my virtual machine on Google Cloud to enable seamless SSH access using my regular terminal. I have already set up a virtual machine on Google Cloud, but I am ...
0
votes
1
answer
34
views
GCP: can we use output of one GSUTIL command embedded in other GSUTIL command
I have a requirement where have a file in GCS and I need to remove the header and trailer of a file and load the data into other GCS bucket.
I have tried using something like this, but its not working....
0
votes
0
answers
28
views
Gen2 Cloud Functions Triggered by Pub/Sub and HTTPS - Receiving Undefined Request in HTTP Trigger
I have a Google Cloud Function (Gen2) that is primarily triggered by Pub/Sub messages. I want to extend its functionality so that it can also be triggered by HTTPS requests. However, I'm encountering ...
1
vote
1
answer
28
views
Hadoop fs configurations in Dataproc spark code
I came across a spark code, which runs on GCP dataproc , reading and writing data to GCS. The code has below spark configurations.
spark_session.sparkContext._conf.set("fs.gs.impl", "...
0
votes
0
answers
33
views
How to export BigQuery table to parquet using SQL and keep columns mode
I'm trying to export BQ table to GCS in the parquet format. Can't use API because of the size limits, so it's gotta be SQL query. The problem is when using SQL query, parquet ignores table columns ...
0
votes
0
answers
15
views
Cloud function: doc.pipe(fs.createWriteStream("output.pdf")) doesn't work with api request
I am using google cloud functions to run an api with node js. I am using pdfkit to create a pdf.
When I run node index.js (in the terminal) manually. The pdf gets generated. When I do the api request ...
0
votes
0
answers
17
views
Speeding up INSERT Query of a Pandas Dataframe into MySQL Database
I am using simple to_sql pandas method to save dataframe into MySQL database:
engine = create_engine('mysql+mysqlconnector://xxxx:xxxxx@localhost:3306/db_qa')
df.to_sql('my_table', con=engine, ...
0
votes
1
answer
35
views
How to remove Firebase project without deleting entire GCP project?
I originally had a Google Cloud project with some APIs enabled for Maps and a few other services.
Later, I wanted to integrate Firebase Cloud Messaging, so I created a new Firebase project and linked ...
0
votes
0
answers
18
views
How can I use the google-cloud vision API in my browser-based web app
I am making a website with a p5 canvas that uses your webcam and displays it on the canvas. I am using the ml5 image classifier of MobileNet, and I also want it to display the results from google ...
0
votes
1
answer
15
views
Unable to give particular user access to a particular object in GCP bucket
Need to give access to a particular user to access a particular bucket object in gcp bucket storage.
Help with the IAM policy and what will be the steps for giving this permission.
I have tried with ...
0
votes
0
answers
30
views
Google Cloud API php authentication fails
I closely followed the docs
https://github.com/googleapis/google-api-php-client/blob/main/docs/oauth-server.md
https://developers.google.com/android-publisher/getting_started?hl=en
Created an service ...
0
votes
1
answer
29
views
How to Stream data with Airflow
I have a very large csv files. I want to stream the rows of data either "one by one" or "batch by batch" and put (WRITE_APPEND) it into BigQuery using AirFlow. How can i achieve ...
0
votes
0
answers
11
views
Is it possible for GFE to use a managed certs to establish TLS with mesh ingress on GKE?
I'm trying ASM(Anthos Service Mesh) to create a service mesh in the GKE.
I'm looking at the following steps, but do we need to use a self-signed certificate here?
In the following steps, you generate ...
0
votes
1
answer
16
views
Does creating a compute_disk in gcp with a custom image incur additional costs for the image storage?
I would like to create a google_compute_disk resource through terraform. If I want to create the disk with a custom image that is not from the ones provided by google, do I have to pay for storage for ...
0
votes
0
answers
47
views
Does Cloud Search API require cloud search to be signed up?
I need to use Google Cloud Search API for using with non-google workspace data (third party data like PDF etc.)
Is it mandatory to sign up for cloud search to use cloud search API?
Right now don't ...
0
votes
1
answer
12
views
Transcoding video to mp3: config.muxStreams[0].container is not supported, supported value from [ts fmp4 mp4 vtt]
I want to convert my mp4 file on Cloud Storage into mp3 file using Transcoder API.
Official page says it supports mp3 file and its codec. https://cloud.google.com/transcoder/docs/concepts/supported-...
0
votes
0
answers
47
views
Extend the Core method chain in DataForm
We are developing a model in DataForm using Javascript.
https://cloud.google.com/dataform/docs/develop-workflows-js?hl=ja
We would like to extend some behaviors, for the domain we want to use.
More ...