Involved Source Filesacl.gobucket.gocopy.go
Package storage provides an easy way to work with Google Cloud Storage.
Google Cloud Storage stores data in named objects, which are grouped into buckets.
More information about Google Cloud Storage is available at
https://cloud.google.com/storage/docs.
See https://godoc.org/cloud.google.com/go for authentication, timeouts,
connection pooling and similar aspects of this package.
All of the methods of this package use exponential backoff to retry calls that fail
with certain errors, as described in
https://cloud.google.com/storage/docs/exponential-backoff. Retrying continues
indefinitely unless the controlling context is canceled or the client is closed. See
context.WithTimeout and context.WithCancel.
Creating a Client
To start working with this package, create a client:
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: Handle error.
}
The client will use your default application credentials.
If you only wish to access public data, you can create
an unauthenticated client with
client, err := storage.NewClient(ctx, option.WithoutAuthentication())
Buckets
A Google Cloud Storage bucket is a collection of objects. To work with a
bucket, make a bucket handle:
bkt := client.Bucket(bucketName)
A handle is a reference to a bucket. You can have a handle even if the
bucket doesn't exist yet. To create a bucket in Google Cloud Storage,
call Create on the handle:
if err := bkt.Create(ctx, projectID, nil); err != nil {
// TODO: Handle error.
}
Note that although buckets are associated with projects, bucket names are
global across all projects.
Each bucket has associated metadata, represented in this package by
BucketAttrs. The third argument to BucketHandle.Create allows you to set
the initial BucketAttrs of a bucket. To retrieve a bucket's attributes, use
Attrs:
attrs, err := bkt.Attrs(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("bucket %s, created at %s, is located in %s with storage class %s\n",
attrs.Name, attrs.Created, attrs.Location, attrs.StorageClass)
Objects
An object holds arbitrary data as a sequence of bytes, like a file. You
refer to objects using a handle, just as with buckets, but unlike buckets
you don't explicitly create an object. Instead, the first time you write
to an object it will be created. You can use the standard Go io.Reader
and io.Writer interfaces to read and write object data:
obj := bkt.Object("data")
// Write something to obj.
// w implements io.Writer.
w := obj.NewWriter(ctx)
// Write some text to obj. This will either create the object or overwrite whatever is there already.
if _, err := fmt.Fprintf(w, "This object contains text.\n"); err != nil {
// TODO: Handle error.
}
// Close, just like writing a file.
if err := w.Close(); err != nil {
// TODO: Handle error.
}
// Read it back.
r, err := obj.NewReader(ctx)
if err != nil {
// TODO: Handle error.
}
defer r.Close()
if _, err := io.Copy(os.Stdout, r); err != nil {
// TODO: Handle error.
}
// Prints "This object contains text."
Objects also have attributes, which you can fetch with Attrs:
objAttrs, err := obj.Attrs(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("object %s has size %d and can be read using %s\n",
objAttrs.Name, objAttrs.Size, objAttrs.MediaLink)
Listing objects
Listing objects in a bucket is done with the Bucket.Objects method:
query := &storage.Query{Prefix: ""}
var names []string
it := bkt.Objects(ctx, query)
for {
attrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatal(err)
}
names = append(names, attrs.Name)
}
If only a subset of object attributes is needed when listing, specifying this
subset using Query.SetAttrSelection may speed up the listing process:
query := &storage.Query{Prefix: ""}
query.SetAttrSelection([]string{"Name"})
// ... as before
ACLs
Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of
ACLRules, each of which specifies the role of a user, group or project. ACLs
are suitable for fine-grained control, but you may prefer using IAM to control
access at the project level (see
https://cloud.google.com/storage/docs/access-control/iam).
To list the ACLs of a bucket or object, obtain an ACLHandle and call its List method:
acls, err := obj.ACL().List(ctx)
if err != nil {
// TODO: Handle error.
}
for _, rule := range acls {
fmt.Printf("%s has role %s\n", rule.Entity, rule.Role)
}
You can also set and delete ACLs.
Conditions
Every object has a generation and a metageneration. The generation changes
whenever the content changes, and the metageneration changes whenever the
metadata changes. Conditions let you check these values before an operation;
the operation only executes if the conditions match. You can use conditions to
prevent race conditions in read-modify-write operations.
For example, say you've read an object's metadata into objAttrs. Now
you want to write to that object, but only if its contents haven't changed
since you read it. Here is how to express that:
w = obj.If(storage.Conditions{GenerationMatch: objAttrs.Generation}).NewWriter(ctx)
// Proceed with writing as above.
Signed URLs
You can obtain a URL that lets anyone read or write an object for a limited time.
You don't need to create a client to do this. See the documentation of
SignedURL for details.
url, err := storage.SignedURL(bucketName, "shared-object", opts)
if err != nil {
// TODO: Handle error.
}
fmt.Println(url)
Post Policy V4 Signed Request
A type of signed request that allows uploads through HTML forms directly to Cloud Storage with
temporary permission. Conditions can be applied to restrict how the HTML form is used and exercised
by a user.
For more information, please see https://cloud.google.com/storage/docs/xml-api/post-object as well
as the documentation of GenerateSignedPostPolicyV4.
pv4, err := storage.GenerateSignedPostPolicyV4(bucketName, objectName, opts)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("URL: %s\nFields; %v\n", pv4.URL, pv4.Fields)
Errors
Errors returned by this client are often of the type [`googleapi.Error`](https://godoc.org/google.golang.org/api/googleapi#Error).
These errors can be introspected for more information by type asserting to the richer `googleapi.Error` type. For example:
if e, ok := err.(*googleapi.Error); ok {
if e.Code == 409 { ... }
}
go110.gohmac.goiam.goinvoke.gonotifications.gopost_policy_v4.goreader.gostorage.gowriter.go
Code Examples
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// No longer grant access to the bucket to everyone on the Internet.
if err := client.Bucket("my-bucket").ACL().Delete(ctx, storage.AllUsers); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// List the default object ACLs for my-bucket.
aclRules, err := client.Bucket("my-bucket").DefaultObjectACL().List(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(aclRules)
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Let any authenticated user read my-bucket/my-object.
obj := client.Bucket("my-bucket").Object("my-object")
if err := obj.ACL().Set(ctx, storage.AllAuthenticatedUsers, storage.RoleReader); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
n, err := b.AddNotification(ctx, &storage.Notification{
TopicProjectID: "my-project",
TopicID: "my-topic",
PayloadFormat: storage.JSONPayload,
})
if err != nil {
// TODO: handle error.
}
fmt.Println(n.ID)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
if err := client.Bucket("my-bucket").Create(ctx, "my-project", nil); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
if err := client.Bucket("my-bucket").Delete(ctx); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
var notificationID string
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
// TODO: Obtain notificationID from BucketHandle.AddNotification
// or BucketHandle.Notifications.
err = b.DeleteNotification(ctx, notificationID)
if err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
attrs, err := b.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
// Note that locking the bucket without first attaching a RetentionPolicy
// that's at least 1 day is a no-op
err = b.If(storage.BucketConditions{MetagenerationMatch: attrs.MetaGeneration}).LockRetentionPolicy(ctx)
if err != nil {
// TODO: handle err
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
ns, err := b.Notifications(ctx)
if err != nil {
// TODO: handle error.
}
for id, n := range ns {
fmt.Printf("%s: %+v\n", id, n)
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Bucket("my-bucket").Objects(ctx, nil)
_ = it // TODO: iterate using Next or iterator.Pager.
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Enable versioning in the bucket, regardless of its previous value.
attrs, err := client.Bucket("my-bucket").Update(ctx,
storage.BucketAttrsToUpdate{VersioningEnabled: true})
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
attrs, err := b.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
var au storage.BucketAttrsToUpdate
au.SetLabel("lab", attrs.Labels["lab"]+"-more")
if attrs.Labels["delete-me"] == "yes" {
au.DeleteLabel("delete-me")
}
attrs, err = b.
If(storage.BucketConditions{MetagenerationMatch: attrs.MetaGeneration}).
Update(ctx, au)
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Buckets(ctx, "my-project")
for {
bucketAttrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: Handle error.
}
fmt.Println(bucketAttrs)
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Buckets(ctx, "my-bucket")
_ = it // TODO: iterate using Next or iterator.Pager.
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkey, err := client.CreateHMACKey(ctx, "project-id", "service-account-email")
if err != nil {
// TODO: handle error.
}
_ = hkey // TODO: Use the HMAC Key.
}
package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id")
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id", storage.ForHMACKeyServiceAccountEmail("service@account.email"))
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id", storage.ShowDeletedHMACKeys())
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
bkt := client.Bucket("bucketname")
src1 := bkt.Object("o1")
src2 := bkt.Object("o2")
dst := bkt.Object("o3")
// Compose and modify metadata.
c := dst.ComposerFrom(src1, src2)
c.ContentType = "text/plain"
// Set the expected checksum for the destination object to be validated by
// the backend (if desired).
c.CRC32C = 42
c.SendCRC32C = true
attrs, err := c.Run(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Println(attrs)
// Just compose.
attrs, err = dst.ComposerFrom(src1, src2).Run(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Println(attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
src := client.Bucket("bucketname").Object("file1")
dst := client.Bucket("another-bucketname").Object("file2")
// Copy content and modify metadata.
copier := dst.CopierFrom(src)
copier.ContentType = "text/plain"
attrs, err := copier.Run(ctx)
if err != nil {
// TODO: Handle error, possibly resuming with copier.RewriteToken.
}
fmt.Println(attrs)
// Just copy content.
attrs, err = dst.CopierFrom(src).Run(ctx)
if err != nil {
// TODO: Handle error. No way to resume.
}
fmt.Println(attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"log"
)
func main() {
// Display progress across multiple rewrite RPCs.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
src := client.Bucket("bucketname").Object("file1")
dst := client.Bucket("another-bucketname").Object("file2")
copier := dst.CopierFrom(src)
copier.ProgressFunc = func(copiedBytes, totalBytes uint64) {
log.Printf("copy %.1f%% done", float64(copiedBytes)/float64(totalBytes)*100)
}
if _, err := copier.Run(ctx); err != nil {
// TODO: handle error.
}
}
package main
import (
"bytes"
"cloud.google.com/go/storage"
"io"
"mime/multipart"
"net/http"
"time"
)
func main() {
pv4, err := storage.GenerateSignedPostPolicyV4("my-bucket", "my-object.txt", &storage.PostPolicyV4Options{
GoogleAccessID: "my-access-id",
PrivateKey: []byte("my-private-key"),
// The upload expires in 2hours.
Expires: time.Now().Add(2 * time.Hour),
Fields: &storage.PolicyV4Fields{
StatusCodeOnSuccess: 200,
RedirectToURLOnSuccess: "https://example.org/",
// It MUST only be a text file.
ContentType: "text/plain",
},
// The conditions that the uploaded file will be expected to conform to.
Conditions: []storage.PostPolicyV4Condition{
// Make the file a maximum of 10mB.
storage.ConditionContentLengthRange(0, 10<<20),
},
})
if err != nil {
// TODO: handle error.
}
// Now you can upload your file using the generated post policy
// with a plain HTTP client or even the browser.
formBuf := new(bytes.Buffer)
mw := multipart.NewWriter(formBuf)
for fieldName, value := range pv4.Fields {
if err := mw.WriteField(fieldName, value); err != nil {
// TODO: handle error.
}
}
file := bytes.NewReader(bytes.Repeat([]byte("a"), 100))
mf, err := mw.CreateFormFile("file", "myfile.txt")
if err != nil {
// TODO: handle error.
}
if _, err := io.Copy(mf, file); err != nil {
// TODO: handle error.
}
if err := mw.Close(); err != nil {
// TODO: handle error.
}
// Compose the request.
req, err := http.NewRequest("POST", pv4.URL, formBuf)
if err != nil {
// TODO: handle error.
}
// Ensure the Content-Type is derived from the multipart writer.
req.Header.Set("Content-Type", mw.FormDataContentType())
res, err := http.DefaultClient.Do(req)
if err != nil {
// TODO: handle error.
}
_ = res
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
// Make sure that the HMACKey being deleted has a status of inactive.
if err := hkh.Delete(ctx); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
hkey, err := hkh.Get(ctx)
if err != nil {
// TODO: handle error.
}
_ = hkey // TODO: Use the HMAC Key.
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
ukey, err := hkh.Update(ctx, storage.HMACKeyAttrsToUpdate{
State: storage.Inactive,
})
if err != nil {
// TODO: handle error.
}
_ = ukey // TODO: Use the HMAC Key.
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
// Use Google Application Default Credentials to authorize and authenticate the client.
// More information about Application Default Credentials and how to enable is at
// https://developers.google.com/identity/protocols/application-default-credentials.
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Use the client.
// Close the client when finished.
if err := client.Close(); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/option"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx, option.WithoutAuthentication())
if err != nil {
// TODO: handle error.
}
// Use the client.
// Close the client when finished.
if err := client.Close(); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
objAttrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
// Read the object.
objAttrs1, err := obj.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
// Do something else for a while.
time.Sleep(5 * time.Minute)
// Now read the same contents, even if the object has been written since the last read.
objAttrs2, err := obj.Generation(objAttrs1.Generation).Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs1, objAttrs2)
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
var key1, key2 []byte
func main() {
// To rotate the encryption key on an object, copy it onto itself.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("bucketname").Object("obj")
// Assume obj is encrypted with key1, and we want to change to key2.
_, err = obj.Key(key2).CopierFrom(obj.Key(key1)).Run(ctx)
if err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// To delete multiple objects in a bucket, list them with an
// ObjectIterator, then Delete them.
// If you are using this package on the App Engine Flex runtime,
// you can init a bucket client with your app's default bucket name.
// See http://godoc.org/google.golang.org/appengine/file#DefaultBucketName.
bucket := client.Bucket("my-bucket")
it := bucket.Objects(ctx, nil)
for {
objAttrs, err := it.Next()
if err != nil && err != iterator.Done {
// TODO: Handle error.
}
if err == iterator.Done {
break
}
if err := bucket.Object(objAttrs.Name).Delete(ctx); err != nil {
// TODO: Handle error.
}
}
fmt.Println("deleted all object items in the bucket specified.")
}
package main
import (
"cloud.google.com/go/storage"
"context"
"io"
"os"
)
var gen int64
func main() {
// Read an object's contents from generation gen, regardless of the
// current generation of the object.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
rc, err := obj.Generation(gen).NewReader(ctx)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
if _, err := io.Copy(os.Stdout, rc); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/googleapi"
"io"
"net/http"
"os"
)
var gen int64
func main() {
// Read from an object only if the current generation is gen.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
rc, err := obj.If(storage.Conditions{GenerationMatch: gen}).NewReader(ctx)
if err != nil {
// TODO: handle error.
}
if _, err := io.Copy(os.Stdout, rc); err != nil {
// TODO: handle error.
}
if err := rc.Close(); err != nil {
switch ee := err.(type) {
case *googleapi.Error:
if ee.Code == http.StatusPreconditionFailed {
// The condition presented in the If failed.
// TODO: handle error.
}
// TODO: handle other status codes here.
default:
// TODO: handle error.
}
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
var secretKey []byte
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
// Encrypt the object's contents.
w := obj.Key(secretKey).NewWriter(ctx)
if _, err := w.Write([]byte("top secret")); err != nil {
// TODO: handle error.
}
if err := w.Close(); err != nil {
// TODO: handle error.
}
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read only the first 64K.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, 0, 64*1024)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("first 64K of file contents:\n%s\n", slurp)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read only the last 10 bytes until the end of the file.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, -10, -1)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("Last 10 bytes from the end of the file:\n%s\n", slurp)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read from the 101st byte until the end of the file.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, 100, -1)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("From 101st byte until the end:\n%s\n", slurp)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
rc, err := client.Bucket("my-bucket").Object("my-object").NewReader(ctx)
if err != nil {
// TODO: handle error.
}
slurp, err := ioutil.ReadAll(rc)
rc.Close()
if err != nil {
// TODO: handle error.
}
fmt.Println("file contents:", slurp)
}
package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
_ = wc // TODO: Use the Writer.
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Change only the content type of the object.
objAttrs, err := client.Bucket("my-bucket").Object("my-object").Update(ctx, storage.ObjectAttrsToUpdate{
ContentType: "text/html",
ContentDisposition: "", // delete ContentDisposition
})
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Bucket("my-bucket").Objects(ctx, nil)
for {
objAttrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: Handle error.
}
fmt.Println(objAttrs)
}
}
package main
import (
"cloud.google.com/go/storage"
"fmt"
"io/ioutil"
"time"
)
func main() {
pkey, err := ioutil.ReadFile("my-private-key.pem")
if err != nil {
// TODO: handle error.
}
url, err := storage.SignedURL("my-bucket", "my-object", &storage.SignedURLOptions{
GoogleAccessID: "xxx@developer.gserviceaccount.com",
PrivateKey: pkey,
Method: "GET",
Expires: time.Now().Add(48 * time.Hour),
})
if err != nil {
// TODO: handle error.
}
fmt.Println(url)
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
wc.ContentType = "text/plain"
wc.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"hash/crc32"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
data := []byte("verify me")
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
wc.CRC32C = crc32.Checksum(data, crc32.MakeTable(crc32.Castagnoli))
wc.SendCRC32C = true
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
tctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel() // Cancel when done, whether we time out or not.
wc := client.Bucket("bucketname").Object("filename1").NewWriter(tctx)
wc.ContentType = "text/plain"
wc.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
Package-Level Type Names (total 58, in which 48 are exported)
BucketAttrs represents the metadata for a Google Cloud Storage bucket.
Read-only fields are ignored by BucketHandle.Create.
ACL is the list of access control rules on the bucket.
BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
UniformBucketLevelAccess is recommended above the use of this field.
Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
true, will enable UniformBucketLevelAccess.
The bucket's Cross-Origin Resource Sharing (CORS) configuration.
Created is the creation time of the bucket.
This field is read-only.
DefaultEventBasedHold is the default value for event-based hold on
newly created objects in this bucket. It defaults to false.
DefaultObjectACL is the list of access controls to
apply to new objects when no object ACL is provided.
The encryption configuration used by default for newly inserted objects.
Etag is the HTTP/1.1 Entity tag for the bucket.
This field is read-only.
Labels are the bucket's labels.
Lifecycle is the lifecycle configuration for objects in the bucket.
Location is the location of the bucket. It defaults to "US".
LocationType describes how data is stored and replicated.
Typical values are "multi-region", "region" and "dual-region".
This field is read-only.
The logging configuration.
MetaGeneration is the metadata generation of the bucket.
This field is read-only.
Name is the name of the bucket.
This field is read-only.
If not empty, applies a predefined set of access controls. It should be set
only when creating a bucket.
It is always empty for BucketAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
for valid values.
If not empty, applies a predefined set of default object access controls.
It should be set only when creating a bucket.
It is always empty for BucketAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
for valid values.
RequesterPays reports whether the bucket is a Requester Pays bucket.
Clients performing operations on Requester Pays buckets must provide
a user project (see BucketHandle.UserProject), which will be billed
for the operations.
Retention policy enforces a minimum retention time for all objects
contained in the bucket. A RetentionPolicy of nil implies the bucket
has no minimum data retention.
This feature is in private alpha release. It is not currently available to
most customers. It might be changed in backwards-incompatible ways and is not
subject to any SLA or deprecation policy.
StorageClass is the default storage class of the bucket. This defines
how objects in the bucket are stored and determines the SLA
and the cost of storage. Typical values are "STANDARD", "NEARLINE",
"COLDLINE" and "ARCHIVE". Defaults to "STANDARD".
See https://cloud.google.com/storage/docs/storage-classes for all
valid values.
UniformBucketLevelAccess configures access checks to use only bucket-level IAM
policies and ignore any ACL rules for the bucket.
See https://cloud.google.com/storage/docs/uniform-bucket-level-access
for more information.
VersioningEnabled reports whether this bucket has versioning enabled.
The website configuration.
toRawBucket copies the editable attribute from b to the raw library's Bucket type.
func (*BucketHandle).Attrs(ctx context.Context) (attrs *BucketAttrs, err error)
func (*BucketHandle).Update(ctx context.Context, uattrs BucketAttrsToUpdate) (attrs *BucketAttrs, err error)
func (*BucketIterator).Next() (*BucketAttrs, error)
func newBucket(b *raw.Bucket) (*BucketAttrs, error)
func (*BucketHandle).Create(ctx context.Context, projectID string, attrs *BucketAttrs) (err error)
BucketAttrsToUpdate define the attributes to update during an Update call.
BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
UniformBucketLevelAccess is recommended above the use of this field.
Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
true, will enable UniformBucketLevelAccess. If both BucketPolicyOnly and
UniformBucketLevelAccess are set, the value of UniformBucketLevelAccess
will take precedence.
If set, replaces the CORS configuration with a new configuration.
An empty (rather than nil) slice causes all CORS policies to be removed.
DefaultEventBasedHold is the default value for event-based hold on
newly created objects in this bucket.
If set, replaces the encryption configuration of the bucket. Using
BucketEncryption.DefaultKMSKeyName = "" will delete the existing
configuration.
If set, replaces the lifecycle configuration of the bucket.
If set, replaces the logging configuration of the bucket.
If not empty, applies a predefined set of access controls.
See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
If not empty, applies a predefined set of default object access controls.
See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
If set, updates whether the bucket is a Requester Pays bucket.
If set, updates the retention policy of the bucket. Using
RetentionPolicy.RetentionPeriod = 0 will delete the existing policy.
This feature is in private alpha release. It is not currently available to
most customers. It might be changed in backwards-incompatible ways and is not
subject to any SLA or deprecation policy.
UniformBucketLevelAccess configures access checks to use only bucket-level IAM
policies and ignore any ACL rules for the bucket.
See https://cloud.google.com/storage/docs/uniform-bucket-level-access
for more information.
If set, updates whether the bucket uses versioning.
If set, replaces the website configuration of the bucket.
deleteLabelsmap[string]boolsetLabelsmap[string]string
DeleteLabel causes a label to be deleted when ua is used in a
call to Bucket.Update.
SetLabel causes a label to be added or modified when ua is used
in a call to Bucket.Update.
(*T) toRawBucket() *raw.Bucket
func (*BucketHandle).Update(ctx context.Context, uattrs BucketAttrsToUpdate) (attrs *BucketAttrs, err error)
func (*BucketHandle).newPatchCall(uattrs *BucketAttrsToUpdate) (*raw.BucketsPatchCall, error)
BucketConditions constrain bucket methods to act on specific metagenerations.
The zero value is an empty set of constraints.
MetagenerationMatch specifies that the bucket must have the given
metageneration for the operation to occur.
If MetagenerationMatch is zero, it has no effect.
MetagenerationNotMatch specifies that the bucket must not have the given
metageneration for the operation to occur.
If MetagenerationNotMatch is zero, it has no effect.
(*T) validate(method string) error
func (*BucketHandle).If(conds BucketConditions) *BucketHandle
func applyBucketConds(method string, conds *BucketConditions, call interface{}) error
BucketEncryption is a bucket's encryption configuration.
A Cloud KMS key name, in the form
projects/P/locations/L/keyRings/R/cryptoKeys/K, that will be used to encrypt
objects inserted into this bucket, if no encryption method is specified.
The key's location must be the same as the bucket's.
(*T) toRawBucketEncryption() *raw.BucketEncryption
func toBucketEncryption(e *raw.BucketEncryption) *BucketEncryption
BucketHandle provides operations on a Google Cloud Storage bucket.
Use Client.Bucket to get a handle.
aclACLHandlec*Clientconds*BucketConditionsdefaultObjectACLACLHandlenamestring
// project for Requester Pays buckets
ACL returns an ACLHandle, which provides access to the bucket's access control list.
This controls who can list, create or overwrite the objects in a bucket.
This call does not perform any network operations.
AddNotification adds a notification to b. You must set n's TopicProjectID, TopicID
and PayloadFormat, and must not set its ID. The other fields are all optional. The
returned Notification's ID can be used to refer to it.
Attrs returns the metadata for the bucket.
Create creates the Bucket in the project.
If attrs is nil the API defaults will be used.
DefaultObjectACL returns an ACLHandle, which provides access to the bucket's default object ACLs.
These ACLs are applied to newly created objects in this bucket that do not have a defined ACL.
This call does not perform any network operations.
Delete deletes the Bucket.
DeleteNotification deletes the notification with the given ID.
IAM provides access to IAM access control for the bucket.
If returns a new BucketHandle that applies a set of preconditions.
Preconditions already set on the BucketHandle are ignored.
Operations on the new handle will return an error if the preconditions are not
satisfied. The only valid preconditions for buckets are MetagenerationMatch
and MetagenerationNotMatch.
LockRetentionPolicy locks a bucket's retention policy until a previously-configured
RetentionPeriod past the EffectiveTime. Note that if RetentionPeriod is set to less
than a day, the retention policy is treated as a development configuration and locking
will have no effect. The BucketHandle must have a metageneration condition that
matches the bucket's metageneration. See BucketHandle.If.
This feature is in private alpha release. It is not currently available to
most customers. It might be changed in backwards-incompatible ways and is not
subject to any SLA or deprecation policy.
Notifications returns all the Notifications configured for this bucket, as a map
indexed by notification ID.
Object returns an ObjectHandle, which provides operations on the named object.
This call does not perform any network operations.
name must consist entirely of valid UTF-8-encoded runes. The full specification
for valid object names can be found at:
https://cloud.google.com/storage/docs/bucket-naming
Objects returns an iterator over the objects in the bucket that match the Query q.
If q is nil, no filtering is done.
Note: The returned iterator is not safe for concurrent operations without explicit synchronization.
Update updates a bucket's attributes.
UserProject returns a new BucketHandle that passes the project ID as the user
project for all subsequent calls. Calls with a user project will be billed to that
project rather than to the bucket's owning project.
A user project is required for all operations on Requester Pays buckets.
(*T) newDeleteCall() (*raw.BucketsDeleteCall, error)(*T) newGetCall() (*raw.BucketsGetCall, error)(*T) newPatchCall(uattrs *BucketAttrsToUpdate) (*raw.BucketsPatchCall, error)
func (*BucketHandle).If(conds BucketConditions) *BucketHandle
func (*BucketHandle).UserProject(projectID string) *BucketHandle
func (*Client).Bucket(name string) *BucketHandle
A BucketIterator is an iterator over BucketAttrs.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
Prefix restricts the iterator to buckets whose names begin with it.
buckets[]*BucketAttrsclient*Clientctxcontext.ContextnextFuncfunc() errorpageInfo*iterator.PageInfoprojectIDstring
Next returns the next result. Its second return value is iterator.Done if
there are no more results. Once Next returns iterator.Done, all subsequent
calls will return iterator.Done.
Note: This method is not safe for concurrent operations without explicit synchronization.
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This method is not safe for concurrent operations without explicit synchronization.
(*T) fetch(pageSize int, pageToken string) (token string, err error)
*T : google.golang.org/api/iterator.Pageable
func (*Client).Buckets(ctx context.Context, projectID string) *BucketIterator
BucketLogging holds the bucket's logging configuration, which defines the
destination bucket and optional name prefix for the current bucket's
logs.
The destination bucket where the current bucket's logs
should be placed.
A prefix for log object names.
(*T) toRawBucketLogging() *raw.BucketLogging
func toBucketLogging(b *raw.BucketLogging) *BucketLogging
BucketPolicyOnly is an alias for UniformBucketLevelAccess.
Use of UniformBucketLevelAccess is preferred above BucketPolicyOnly.
Enabled specifies whether access checks use only bucket-level IAM
policies. Enabled may be disabled until the locked time.
LockedTime specifies the deadline for changing Enabled from true to
false.
func toBucketPolicyOnly(b *raw.BucketIamConfiguration) BucketPolicyOnly
BucketWebsite holds the bucket's website configuration, controlling how the
service behaves when accessing bucket contents as a web site. See
https://cloud.google.com/storage/docs/static-website for more information.
If the requested object path is missing, the service will ensure the path has
a trailing '/', append this suffix, and attempt to retrieve the resulting
object. This allows the creation of index.html objects to represent directory
pages.
If the requested object path is missing, and any mainPageSuffix object is
missing, if applicable, the service will return the named object from this
bucket as the content for a 404 Not Found result.
(*T) toRawBucketWebsite() *raw.BucketWebsite
func toBucketWebsite(w *raw.BucketWebsite) *BucketWebsite
Client is a client for interacting with Google Cloud Storage.
Clients should be reused instead of created as needed.
The methods of Client are safe for concurrent use by multiple goroutines.
EnvHost is the host set on the STORAGE_EMULATOR_HOST variable.
hc*http.Clientraw*raw.Service
ReadHost is the default host used on the reader.
Scheme describes the scheme under the current host.
Bucket returns a BucketHandle, which provides operations on the named bucket.
This call does not perform any network operations.
The supplied name must contain only lowercase letters, numbers, dashes,
underscores, and dots. The full specification for valid bucket names can be
found at:
https://cloud.google.com/storage/docs/bucket-naming
Buckets returns an iterator over the buckets in the project. You may
optionally set the iterator's Prefix field to restrict the list to buckets
whose names begin with the prefix. By default, all buckets in the project
are returned.
Note: The returned iterator is not safe for concurrent operations without explicit synchronization.
Close closes the Client.
Close need not be called at program exit.
CreateHMACKey invokes an RPC for Google Cloud Storage to create a new HMACKey.
This method is EXPERIMENTAL and subject to change or removal without notice.
HMACKeyHandle creates a handle that will be used for HMACKey operations.
This method is EXPERIMENTAL and subject to change or removal without notice.
ListHMACKeys returns an iterator for listing HMACKeys.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
ServiceAccount fetches the email address of the given project's Google Cloud Storage service account.
*T : io.Closer
func NewClient(ctx context.Context, opts ...option.ClientOption) (*Client, error)
A Composer composes source objects into a destination object.
For Requester Pays buckets, the user project of dst is billed.
ObjectAttrs are optional attributes to set on the destination object.
Any attributes must be initialized before any calls on the Composer. Nil
or zero-valued attributes are ignored.
ACL is the list of access control rules for the object.
Bucket is the name of the bucket containing this GCS object.
This field is read-only.
CRC32C is the CRC32 checksum of the object's content using the Castagnoli93
polynomial. This field is read-only, except when used from a Writer or
Composer. In those cases, if the SendCRC32C field in the Writer or Composer
is set to is true, the uploaded data is rejected if its CRC32C hash does
not match this field.
CacheControl is the Cache-Control header to be sent in the response
headers when serving the object data.
ContentDisposition is the optional Content-Disposition header of the object
sent in the response headers.
ContentEncoding is the encoding of the object's content.
ContentLanguage is the content language of the object's content.
ContentType is the MIME type of the object's content.
Created is the time the object was created. This field is read-only.
CustomerKeySHA256 is the base64-encoded SHA-256 hash of the
customer-supplied encryption key for the object. It is empty if there is
no customer-supplied encryption key.
See // https://cloud.google.com/storage/docs/encryption for more about
encryption in Google Cloud Storage.
Deleted is the time the object was deleted.
If not deleted, it is the zero value. This field is read-only.
Etag is the HTTP/1.1 Entity tag for the object.
This field is read-only.
EventBasedHold specifies whether an object is under event-based hold. New
objects created in a bucket whose DefaultEventBasedHold is set will
default to that value.
Generation is the generation number of the object's content.
This field is read-only.
Cloud KMS key name, in the form
projects/P/locations/L/keyRings/R/cryptoKeys/K, used to encrypt this object,
if the object is encrypted by such a key.
Providing both a KMSKeyName and a customer-supplied encryption key (via
ObjectHandle.Key) will result in an error when writing an object.
MD5 is the MD5 hash of the object's content. This field is read-only,
except when used from a Writer. If set on a Writer, the uploaded
data is rejected if its MD5 hash does not match this field.
MediaLink is an URL to the object's content. This field is read-only.
Metadata represents user-provided metadata, in key/value pairs.
It can be nil if no metadata is provided.
Metageneration is the version of the metadata for this
object at this generation. This field is used for preconditions
and for detecting changes in metadata. A metageneration number
is only meaningful in the context of a particular generation
of a particular object. This field is read-only.
Name is the name of the object within the bucket.
This field is read-only.
Owner is the owner of the object. This field is read-only.
If non-zero, it is in the form of "user-<userId>".
If not empty, applies a predefined set of access controls. It should be set
only when writing, copying or composing an object. When copying or composing,
it acts as the destinationPredefinedAcl parameter.
PredefinedACL is always empty for ObjectAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/objects/insert
for valid values.
Prefix is set only for ObjectAttrs which represent synthetic "directory
entries" when iterating over buckets using Query.Delimiter. See
ObjectIterator.Next. When set, no other fields in ObjectAttrs will be
populated.
RetentionExpirationTime is a server-determined value that specifies the
earliest time that the object's retention period expires.
This is a read-only field.
Size is the length of the object's content. This field is read-only.
StorageClass is the storage class of the object. This defines
how objects are stored and determines the SLA and the cost of storage.
Typical values are "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
Defaults to "STANDARD".
See https://cloud.google.com/storage/docs/storage-classes for all
valid values.
TemporaryHold specifies whether an object is under temporary hold. While
this flag is set to true, the object is protected against deletion and
overwrites.
Updated is the creation or modification time of the object.
For buckets with versioning enabled, changing an object's
metadata does not change this property. This field is read-only.
SendCRC specifies whether to transmit a CRC32C field. It should be set
to true in addition to setting the Composer's CRC32C field, because zero
is a valid CRC and normally a zero would not be transmitted.
If a CRC32C is sent, and the data in the destination object does not match
the checksum, the compose will be rejected.
dst*ObjectHandlesrcs[]*ObjectHandle
Run performs the compose operation.
toRawObject copies the editable attributes from o to the raw library's Object type.
func (*ObjectHandle).ComposerFrom(srcs ...*ObjectHandle) *Composer
Conditions constrain methods to act on specific generations of
objects.
The zero value is an empty set of constraints. Not all conditions or
combinations of conditions are applicable to all methods.
See https://cloud.google.com/storage/docs/generations-preconditions
for details on how these operate.
DoesNotExist specifies that the object must not exist in the bucket for
the operation to occur.
If DoesNotExist is false, it has no effect.
GenerationMatch specifies that the object must have the given generation
for the operation to occur.
If GenerationMatch is zero, it has no effect.
Use DoesNotExist to specify that the object does not exist in the bucket.
GenerationNotMatch specifies that the object must not have the given
generation for the operation to occur.
If GenerationNotMatch is zero, it has no effect.
MetagenerationMatch specifies that the object must have the given
metageneration for the operation to occur.
If MetagenerationMatch is zero, it has no effect.
MetagenerationNotMatch specifies that the object must not have the given
metageneration for the operation to occur.
If MetagenerationNotMatch is zero, it has no effect.
(*T) isGenerationValid() bool(*T) isMetagenerationValid() bool(*T) validate(method string) error
func (*ObjectHandle).If(conds Conditions) *ObjectHandle
func applyConds(method string, gen int64, conds *Conditions, call interface{}) error
func applySourceConds(gen int64, conds *Conditions, call *raw.ObjectsRewriteCall) error
func conditionsQuery(gen int64, conds *Conditions) string
A Copier copies a source object to a destination.
The Cloud KMS key, in the form projects/P/locations/L/keyRings/R/cryptoKeys/K,
that will be used to encrypt the object. Overrides the object's KMSKeyName, if
any.
Providing both a DestinationKMSKeyName and a customer-supplied encryption key
(via ObjectHandle.Key) on the destination object will result in an error when
Run is called.
ObjectAttrs are optional attributes to set on the destination object.
Any attributes must be initialized before any calls on the Copier. Nil
or zero-valued attributes are ignored.
ACL is the list of access control rules for the object.
Bucket is the name of the bucket containing this GCS object.
This field is read-only.
CRC32C is the CRC32 checksum of the object's content using the Castagnoli93
polynomial. This field is read-only, except when used from a Writer or
Composer. In those cases, if the SendCRC32C field in the Writer or Composer
is set to is true, the uploaded data is rejected if its CRC32C hash does
not match this field.
CacheControl is the Cache-Control header to be sent in the response
headers when serving the object data.
ContentDisposition is the optional Content-Disposition header of the object
sent in the response headers.
ContentEncoding is the encoding of the object's content.
ContentLanguage is the content language of the object's content.
ContentType is the MIME type of the object's content.
Created is the time the object was created. This field is read-only.
CustomerKeySHA256 is the base64-encoded SHA-256 hash of the
customer-supplied encryption key for the object. It is empty if there is
no customer-supplied encryption key.
See // https://cloud.google.com/storage/docs/encryption for more about
encryption in Google Cloud Storage.
Deleted is the time the object was deleted.
If not deleted, it is the zero value. This field is read-only.
Etag is the HTTP/1.1 Entity tag for the object.
This field is read-only.
EventBasedHold specifies whether an object is under event-based hold. New
objects created in a bucket whose DefaultEventBasedHold is set will
default to that value.
Generation is the generation number of the object's content.
This field is read-only.
Cloud KMS key name, in the form
projects/P/locations/L/keyRings/R/cryptoKeys/K, used to encrypt this object,
if the object is encrypted by such a key.
Providing both a KMSKeyName and a customer-supplied encryption key (via
ObjectHandle.Key) will result in an error when writing an object.
MD5 is the MD5 hash of the object's content. This field is read-only,
except when used from a Writer. If set on a Writer, the uploaded
data is rejected if its MD5 hash does not match this field.
MediaLink is an URL to the object's content. This field is read-only.
Metadata represents user-provided metadata, in key/value pairs.
It can be nil if no metadata is provided.
Metageneration is the version of the metadata for this
object at this generation. This field is used for preconditions
and for detecting changes in metadata. A metageneration number
is only meaningful in the context of a particular generation
of a particular object. This field is read-only.
Name is the name of the object within the bucket.
This field is read-only.
Owner is the owner of the object. This field is read-only.
If non-zero, it is in the form of "user-<userId>".
If not empty, applies a predefined set of access controls. It should be set
only when writing, copying or composing an object. When copying or composing,
it acts as the destinationPredefinedAcl parameter.
PredefinedACL is always empty for ObjectAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/objects/insert
for valid values.
Prefix is set only for ObjectAttrs which represent synthetic "directory
entries" when iterating over buckets using Query.Delimiter. See
ObjectIterator.Next. When set, no other fields in ObjectAttrs will be
populated.
RetentionExpirationTime is a server-determined value that specifies the
earliest time that the object's retention period expires.
This is a read-only field.
Size is the length of the object's content. This field is read-only.
StorageClass is the storage class of the object. This defines
how objects are stored and determines the SLA and the cost of storage.
Typical values are "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
Defaults to "STANDARD".
See https://cloud.google.com/storage/docs/storage-classes for all
valid values.
TemporaryHold specifies whether an object is under temporary hold. While
this flag is set to true, the object is protected against deletion and
overwrites.
Updated is the creation or modification time of the object.
For buckets with versioning enabled, changing an object's
metadata does not change this property. This field is read-only.
ProgressFunc can be used to monitor the progress of a multi-RPC copy
operation. If ProgressFunc is not nil and copying requires multiple
calls to the underlying service (see
https://cloud.google.com/storage/docs/json_api/v1/objects/rewrite), then
ProgressFunc will be invoked after each call with the number of bytes of
content copied so far and the total size in bytes of the source object.
ProgressFunc is intended to make upload progress available to the
application. For example, the implementation of ProgressFunc may update
a progress bar in the application's UI, or log the result of
float64(copiedBytes)/float64(totalBytes).
ProgressFunc should return quickly without blocking.
RewriteToken can be set before calling Run to resume a copy
operation. After Run returns a non-nil error, RewriteToken will
have been updated to contain the value needed to resume the copy.
dst*ObjectHandlesrc*ObjectHandle
Run performs the copy.
(*T) callRewrite(ctx context.Context, rawObj *raw.Object) (*raw.RewriteResponse, error)
toRawObject copies the editable attributes from o to the raw library's Object type.
func (*ObjectHandle).CopierFrom(src *ObjectHandle) *Copier
CORS is the bucket's Cross-Origin Resource Sharing (CORS) configuration.
MaxAge is the value to return in the Access-Control-Max-Age
header used in preflight responses.
Methods is the list of HTTP methods on which to include CORS response
headers, (GET, OPTIONS, POST, etc) Note: "*" is permitted in the list
of methods, and means "any method".
Origins is the list of Origins eligible to receive CORS response
headers. Note: "*" is permitted in the list of origins, and means
"any Origin".
ResponseHeaders is the list of HTTP headers other than the simple
response headers to give permission for the user-agent to share
across domains.
func toCORS(rc []*raw.BucketCors) []CORS
func toRawCORS(c []CORS) []*raw.BucketCors
HMACKey is the representation of a Google Cloud Storage HMAC key.
HMAC keys are used to authenticate signed access to objects. To enable HMAC key
authentication, please visit https://cloud.google.com/storage/docs/migrating.
This type is EXPERIMENTAL and subject to change or removal without notice.
AccessID is the ID of the HMAC key.
CreatedTime is the creation time of the HMAC key.
Etag is the HTTP/1.1 Entity tag.
ID is the ID of the HMAC key, including the ProjectID and AccessID.
ProjectID is the ID of the project that owns the
service account to which the key authenticates.
The HMAC's secret key.
ServiceAccountEmail is the email address
of the key's associated service account.
State is the state of the HMAC key.
It can be one of StateActive, StateInactive or StateDeleted.
UpdatedTime is the last modification time of the HMAC key metadata.
func (*Client).CreateHMACKey(ctx context.Context, projectID, serviceAccountEmail string, opts ...HMACKeyOption) (*HMACKey, error)
func (*HMACKeyHandle).Get(ctx context.Context, opts ...HMACKeyOption) (*HMACKey, error)
func (*HMACKeyHandle).Update(ctx context.Context, au HMACKeyAttrsToUpdate, opts ...HMACKeyOption) (*HMACKey, error)
func (*HMACKeysIterator).Next() (*HMACKey, error)
func pbHmacKeyToHMACKey(pb *raw.HmacKey, updatedTimeCanBeNil bool) (*HMACKey, error)
HMACKeyAttrsToUpdate defines the attributes of an HMACKey that will be updated.
This type is EXPERIMENTAL and subject to change or removal without notice.
Etag is an optional field and it is the HTTP/1.1 Entity tag.
State is required and must be either StateActive or StateInactive.
func (*HMACKeyHandle).Update(ctx context.Context, au HMACKeyAttrsToUpdate, opts ...HMACKeyOption) (*HMACKey, error)
HMACKeyHandle helps provide access and management for HMAC keys.
This type is EXPERIMENTAL and subject to change or removal without notice.
accessIDstringprojectIDstringraw*raw.ProjectsHmacKeysService
Delete invokes an RPC to delete the key referenced by accessID, on Google Cloud Storage.
Only inactive HMAC keys can be deleted.
After deletion, a key cannot be used to authenticate requests.
This method is EXPERIMENTAL and subject to change or removal without notice.
Get invokes an RPC to retrieve the HMAC key referenced by the
HMACKeyHandle's accessID.
Options such as UserProjectForHMACKeys can be used to set the
userProject to be billed against for operations.
This method is EXPERIMENTAL and subject to change or removal without notice.
Update mutates the HMACKey referred to by accessID.
This method is EXPERIMENTAL and subject to change or removal without notice.
func (*Client).HMACKeyHandle(projectID, accessID string) *HMACKeyHandle
An HMACKeysIterator is an iterator over HMACKeys.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This type is EXPERIMENTAL and subject to change or removal without notice.
ctxcontext.ContextdeschmacKeyDeschmacKeys[]*HMACKeyindexintnextFuncfunc() errorpageInfo*iterator.PageInfoprojectIDstringraw*raw.ProjectsHmacKeysService
Next returns the next result. Its second return value is iterator.Done if
there are no more results. Once Next returns iterator.Done, all subsequent
calls will return iterator.Done.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
(*T) fetch(pageSize int, pageToken string) (token string, err error)
*T : google.golang.org/api/iterator.Pageable
func (*Client).ListHMACKeys(ctx context.Context, projectID string, opts ...HMACKeyOption) *HMACKeysIterator
HMACState is the state of the HMAC key.
This type is EXPERIMENTAL and subject to change or removal without notice.
const Active
const Deleted
const Inactive
LifecycleAction is a lifecycle configuration action.
StorageClass is the storage class to set on matching objects if the Action
is "SetStorageClass".
Type is the type of action to take on matching objects.
Acceptable values are "Delete" to delete matching objects and
"SetStorageClass" to set the storage class defined in StorageClass on
matching objects.
LifecycleCondition is a set of conditions used to match objects and take an
action automatically.
All configured conditions must be met for the associated action to be taken.
AgeInDays is the age of the object in days.
CreatedBefore is the time the object was created.
This condition is satisfied when an object is created before midnight of
the specified date in UTC.
Liveness specifies the object's liveness. Relevant only for versioned objects
MatchesStorageClasses is the condition matching the object's storage
class.
Values include "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
NumNewerVersions is the condition matching objects with a number of newer versions.
If the value is N, this condition is satisfied when there are at least N
versions (including the live version) newer than this version of the
object.
LifecycleRule is a lifecycle configuration rule.
When all the configured conditions are met by an object in the bucket, the
configured action will automatically be taken on that object.
Action is the action to take when all of the associated conditions are
met.
Condition is the set of conditions that must be met for the associated
action to be taken.
A Notification describes how to send Cloud PubSub messages when certain
events occur in a bucket.
An optional list of additional attributes to attach to each Cloud PubSub
message published for this notification subscription.
Only send notifications about listed event types. If empty, send notifications
for all event types.
See https://cloud.google.com/storage/docs/pubsub-notifications#events.
The ID of the notification.
If present, only apply this notification configuration to object names that
begin with this prefix.
The contents of the message payload.
See https://cloud.google.com/storage/docs/pubsub-notifications#payload.
The ID of the topic to which this subscription publishes.
The ID of the project to which the topic belongs.
func (*BucketHandle).AddNotification(ctx context.Context, n *Notification) (ret *Notification, err error)
func (*BucketHandle).Notifications(ctx context.Context) (n map[string]*Notification, err error)
func notificationsToMap(rns []*raw.Notification) map[string]*Notification
func toNotification(rn *raw.Notification) *Notification
func (*BucketHandle).AddNotification(ctx context.Context, n *Notification) (ret *Notification, err error)
func toRawNotification(n *Notification) *raw.Notification
ObjectAttrs represents the metadata for a Google Cloud Storage (GCS) object.
ACL is the list of access control rules for the object.
Bucket is the name of the bucket containing this GCS object.
This field is read-only.
CRC32C is the CRC32 checksum of the object's content using the Castagnoli93
polynomial. This field is read-only, except when used from a Writer or
Composer. In those cases, if the SendCRC32C field in the Writer or Composer
is set to is true, the uploaded data is rejected if its CRC32C hash does
not match this field.
CacheControl is the Cache-Control header to be sent in the response
headers when serving the object data.
ContentDisposition is the optional Content-Disposition header of the object
sent in the response headers.
ContentEncoding is the encoding of the object's content.
ContentLanguage is the content language of the object's content.
ContentType is the MIME type of the object's content.
Created is the time the object was created. This field is read-only.
CustomerKeySHA256 is the base64-encoded SHA-256 hash of the
customer-supplied encryption key for the object. It is empty if there is
no customer-supplied encryption key.
See // https://cloud.google.com/storage/docs/encryption for more about
encryption in Google Cloud Storage.
Deleted is the time the object was deleted.
If not deleted, it is the zero value. This field is read-only.
Etag is the HTTP/1.1 Entity tag for the object.
This field is read-only.
EventBasedHold specifies whether an object is under event-based hold. New
objects created in a bucket whose DefaultEventBasedHold is set will
default to that value.
Generation is the generation number of the object's content.
This field is read-only.
Cloud KMS key name, in the form
projects/P/locations/L/keyRings/R/cryptoKeys/K, used to encrypt this object,
if the object is encrypted by such a key.
Providing both a KMSKeyName and a customer-supplied encryption key (via
ObjectHandle.Key) will result in an error when writing an object.
MD5 is the MD5 hash of the object's content. This field is read-only,
except when used from a Writer. If set on a Writer, the uploaded
data is rejected if its MD5 hash does not match this field.
MediaLink is an URL to the object's content. This field is read-only.
Metadata represents user-provided metadata, in key/value pairs.
It can be nil if no metadata is provided.
Metageneration is the version of the metadata for this
object at this generation. This field is used for preconditions
and for detecting changes in metadata. A metageneration number
is only meaningful in the context of a particular generation
of a particular object. This field is read-only.
Name is the name of the object within the bucket.
This field is read-only.
Owner is the owner of the object. This field is read-only.
If non-zero, it is in the form of "user-<userId>".
If not empty, applies a predefined set of access controls. It should be set
only when writing, copying or composing an object. When copying or composing,
it acts as the destinationPredefinedAcl parameter.
PredefinedACL is always empty for ObjectAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/objects/insert
for valid values.
Prefix is set only for ObjectAttrs which represent synthetic "directory
entries" when iterating over buckets using Query.Delimiter. See
ObjectIterator.Next. When set, no other fields in ObjectAttrs will be
populated.
RetentionExpirationTime is a server-determined value that specifies the
earliest time that the object's retention period expires.
This is a read-only field.
Size is the length of the object's content. This field is read-only.
StorageClass is the storage class of the object. This defines
how objects are stored and determines the SLA and the cost of storage.
Typical values are "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
Defaults to "STANDARD".
See https://cloud.google.com/storage/docs/storage-classes for all
valid values.
TemporaryHold specifies whether an object is under temporary hold. While
this flag is set to true, the object is protected against deletion and
overwrites.
Updated is the creation or modification time of the object.
For buckets with versioning enabled, changing an object's
metadata does not change this property. This field is read-only.
toRawObject copies the editable attributes from o to the raw library's Object type.
func (*Composer).Run(ctx context.Context) (attrs *ObjectAttrs, err error)
func (*Copier).Run(ctx context.Context) (attrs *ObjectAttrs, err error)
func (*ObjectHandle).Attrs(ctx context.Context) (attrs *ObjectAttrs, err error)
func (*ObjectHandle).Update(ctx context.Context, uattrs ObjectAttrsToUpdate) (oa *ObjectAttrs, err error)
func (*ObjectIterator).Next() (*ObjectAttrs, error)
func (*Writer).Attrs() *ObjectAttrs
func newObject(o *raw.Object) *ObjectAttrs
ObjectHandle provides operations on an object in a Google Cloud Storage bucket.
Use BucketHandle.Object to get a handle.
aclACLHandlebucketstringc*Clientconds*Conditions
// AES-256 key
// a negative value indicates latest
objectstring
// Accept-Encoding: gzip
// for requester-pays buckets
ACL provides access to the object's access control list.
This controls who can read and write this object.
This call does not perform any network operations.
Attrs returns meta information about the object.
ErrObjectNotExist will be returned if the object is not found.
BucketName returns the name of the bucket.
ComposerFrom creates a Composer that can compose srcs into dst.
You can immediately call Run on the returned Composer, or you can
configure it first.
The encryption key for the destination object will be used to decrypt all
source objects and encrypt the destination object. It is an error
to specify an encryption key for any of the source objects.
CopierFrom creates a Copier that can copy src to dst.
You can immediately call Run on the returned Copier, or
you can configure it first.
For Requester Pays buckets, the user project of dst is billed, unless it is empty,
in which case the user project of src is billed.
Delete deletes the single specified object.
Generation returns a new ObjectHandle that operates on a specific generation
of the object.
By default, the handle operates on the latest generation. Not
all operations work when given a specific generation; check the API
endpoints at https://cloud.google.com/storage/docs/json_api/ for details.
If returns a new ObjectHandle that applies a set of preconditions.
Preconditions already set on the ObjectHandle are ignored.
Operations on the new handle will return an error if the preconditions are not
satisfied. See https://cloud.google.com/storage/docs/generations-preconditions
for more details.
Key returns a new ObjectHandle that uses the supplied encryption
key to encrypt and decrypt the object's contents.
Encryption key must be a 32-byte AES-256 key.
See https://cloud.google.com/storage/docs/encryption for details.
NewRangeReader reads part of an object, reading at most length bytes
starting at the given offset. If length is negative, the object is read
until the end. If offset is negative, the object is read abs(offset) bytes
from the end, and length must also be negative to indicate all remaining
bytes will be read.
If the object's metadata property "Content-Encoding" is set to "gzip" or satisfies
decompressive transcoding per https://cloud.google.com/storage/docs/transcoding
that file will be served back whole, regardless of the requested range as
Google Cloud Storage dictates.
NewReader creates a new Reader to read the contents of the
object.
ErrObjectNotExist will be returned if the object is not found.
The caller must call Close on the returned Reader when done reading.
NewWriter returns a storage Writer that writes to the GCS object
associated with this ObjectHandle.
A new object will be created unless an object with this name already exists.
Otherwise any previous object with the same name will be replaced.
The object will not be available (and any previous object will remain)
until Close has been called.
Attributes can be set on the object by modifying the returned Writer's
ObjectAttrs field before the first call to Write. If no ContentType
attribute is specified, the content type will be automatically sniffed
using net/http.DetectContentType.
It is the caller's responsibility to call Close when writing is done. To
stop writing without saving the data, cancel the context.
ObjectName returns the name of the object.
ReadCompressed when true causes the read to happen without decompressing.
Update updates an object with the provided attributes.
All zero-value attributes are ignored.
ErrObjectNotExist will be returned if the object is not found.
(*T) validate() error
func (*BucketHandle).Object(name string) *ObjectHandle
func (*ObjectHandle).Generation(gen int64) *ObjectHandle
func (*ObjectHandle).If(conds Conditions) *ObjectHandle
func (*ObjectHandle).Key(encryptionKey []byte) *ObjectHandle
func (*ObjectHandle).ReadCompressed(compressed bool) *ObjectHandle
func (*ObjectHandle).ComposerFrom(srcs ...*ObjectHandle) *Composer
func (*ObjectHandle).CopierFrom(src *ObjectHandle) *Copier
An ObjectIterator is an iterator over ObjectAttrs.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
bucket*BucketHandlectxcontext.Contextitems[]*ObjectAttrsnextFuncfunc() errorpageInfo*iterator.PageInfoqueryQuery
Next returns the next result. Its second return value is iterator.Done if
there are no more results. Once Next returns iterator.Done, all subsequent
calls will return iterator.Done.
If Query.Delimiter is non-empty, some of the ObjectAttrs returned by Next will
have a non-empty Prefix field, and a zero value for all other fields. These
represent prefixes.
Note: This method is not safe for concurrent operations without explicit synchronization.
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This method is not safe for concurrent operations without explicit synchronization.
(*T) fetch(pageSize int, pageToken string) (string, error)
*T : google.golang.org/api/iterator.Pageable
func (*BucketHandle).Objects(ctx context.Context, q *Query) *ObjectIterator
PolicyV4Fields describes the attributes for a PostPolicyV4 request.
ACL specifies the access control permissions for the object.
Optional.
CacheControl specifies the caching directives for the object.
Optional.
ContentDisposition specifies how the file will be served back to requesters.
Optional.
ContentEncoding specifies the decompressive transcoding that the object.
This field is complementary to ContentType in that the file could be
compressed but ContentType specifies the file's original media type.
Optional.
ContentType specifies the media type of the object.
Optional.
Metadata specifies custom metadata for the object.
If any key doesn't begin with "x-goog-meta-", an error will be returned.
Optional.
RedirectToURLOnSuccess when set, specifies the URL that Cloud Storage
will serve back on successful upload of the object.
Optional.
StatusCodeOnSuccess when set, specifies the status code that Cloud Storage
will serve back on successful upload of the object.
Optional.
PostPolicyV4 describes the URL and respective form fields for a generated PostPolicyV4 request.
Fields specifies the generated key-values that the file uploader
must include in their multipart upload form.
URL is the generated URL that the file upload will be made to.
func GenerateSignedPostPolicyV4(bucket, object string, opts *PostPolicyV4Options) (*PostPolicyV4, error)
PostPolicyV4Options are used to construct a signed post policy.
Please see https://cloud.google.com/storage/docs/xml-api/post-object
for reference about the fields.
The conditions that the uploaded file will be expected to conform to.
When used, the failure of an upload to satisfy a condition will result in
a 4XX status code, back with the message describing the problem.
Optional.
Expires is the expiration time on the signed URL.
It must be a time in the future.
Required.
Fields specifies the attributes of a PostPolicyV4 request.
When Fields is non-nil, its attributes must match those that will
passed into field Conditions.
Optional.
GoogleAccessID represents the authorizer of the signed URL generation.
It is typically the Google service account client email address from
the Google Developers Console in the form of "xxx@developer.gserviceaccount.com".
Required.
Insecure when set indicates that the generated URL's scheme
will use "http" instead of "https" (default).
Optional.
PrivateKey is the Google service account private key. It is obtainable
from the Google Developers Console.
At https://console.developers.google.com/project/<your-project-id>/apiui/credential,
create a service account client ID or reuse one of your existing service account
credentials. Click on the "Generate new P12 key" to generate and download
a new private key. Once you download the P12 file, use the following command
to convert it into a PEM file.
$ openssl pkcs12 -in key.p12 -passin pass:notasecret -out key.pem -nodes
Provide the contents of the PEM file as a byte slice.
Exactly one of PrivateKey or SignBytes must be non-nil.
SignBytes is a function for implementing custom signing. For example, if
your application is running on Google App Engine, you can use
appengine's internal signing function:
ctx := appengine.NewContext(request)
acc, _ := appengine.ServiceAccount(ctx)
url, err := SignedURL("bucket", "object", &SignedURLOptions{
GoogleAccessID: acc,
SignBytes: func(b []byte) ([]byte, error) {
_, signedBytes, err := appengine.SignBytes(ctx, b)
return signedBytes, err
},
// etc.
})
Exactly one of PrivateKey or SignBytes must be non-nil.
Style provides options for the type of URL to use. Options are
PathStyle (default), BucketBoundHostname, and VirtualHostedStyle. See
https://cloud.google.com/storage/docs/request-endpoints for details.
Optional.
func GenerateSignedPostPolicyV4(bucket, object string, opts *PostPolicyV4Options) (*PostPolicyV4, error)
func validatePostPolicyV4Options(opts *PostPolicyV4Options, now time.Time) error
Query represents a query to filter objects from a bucket.
Delimiter returns results in a directory-like fashion.
Results will contain only objects whose names, aside from the
prefix, do not contain delimiter. Objects whose names,
aside from the prefix, contain delimiter will have their name,
truncated after the delimiter, returned in prefixes.
Duplicate prefixes are omitted.
Optional.
Prefix is the prefix filter to query objects
whose names begin with this prefix.
Optional.
Versions indicates whether multiple versions of the same
object will be included in the results.
fieldSelection is used to select only specific fields to be returned by
the query. It's used internally and is populated for the user by
calling Query.SetAttrSelection
SetAttrSelection makes the query populate only specific attributes of
objects. When iterating over objects, if you only need each object's name
and size, pass []string{"Name", "Size"} to this method. Only these fields
will be fetched for each object across the network; the other fields of
ObjectAttr will remain at their default values. This is a performance
optimization; for more information, see
https://cloud.google.com/storage/docs/json_api/v1/how-tos/performance
func (*BucketHandle).Objects(ctx context.Context, q *Query) *ObjectIterator
Reader reads a Cloud Storage object.
It implements io.Reader.
Typically, a Reader computes the CRC of the downloaded content and compares it to
the stored CRC, returning an error from Read if there is a mismatch. This integrity check
is skipped if transcoding occurs. See https://cloud.google.com/storage/docs/transcoding.
AttrsReaderObjectAttrsbodyio.ReadCloser
// should we check the CRC?
// running crc
remainint64reopenfunc(seen int64) (*http.Response, error)seenint64sizeint64
// the CRC32c value the server sent in the header
CacheControl returns the cache control of the object.
Deprecated: use Reader.Attrs.CacheControl.
Close closes the Reader. It must be called when done reading.
ContentEncoding returns the content encoding of the object.
Deprecated: use Reader.Attrs.ContentEncoding.
ContentType returns the content type of the object.
Deprecated: use Reader.Attrs.ContentType.
LastModified returns the value of the Last-Modified header.
Deprecated: use Reader.Attrs.LastModified.
(*T) Read(p []byte) (int, error)
Remain returns the number of bytes left to read, or -1 if unknown.
Size returns the size of the object in bytes.
The returned value is always the same and is not affected by
calls to Read or Close.
Deprecated: use Reader.Attrs.Size.
(*T) readWithRetry(p []byte) (int, error)
*T : github.com/jbenet/go-context/io.Reader
*T : google.golang.org/api/googleapi.ContentTyper
*T : io.Closer
*T : io.ReadCloser
*T : io.Reader
func (*ObjectHandle).NewRangeReader(ctx context.Context, offset, length int64) (r *Reader, err error)
func (*ObjectHandle).NewReader(ctx context.Context) (*Reader, error)
ReaderObjectAttrs are attributes about the object being read. These are populated
during the New call. This struct only holds a subset of object attributes: to
get the full set of attributes, use ObjectHandle.Attrs.
Each field is read-only.
CacheControl specifies whether and for how long browser and Internet
caches are allowed to cache your objects.
ContentEncoding is the encoding of the object's content.
ContentType is the MIME type of the object's content.
Generation is the generation number of the object's content.
LastModified is the time that the object was last modified.
Metageneration is the version of the metadata for this object at
this generation. This field is used for preconditions and for
detecting changes in metadata. A metageneration number is only
meaningful in the context of a particular generation of a
particular object.
Size is the length of the object's content.
StartOffset is the byte offset within the object
from which reading begins.
This value is only non-zero for range requests.
RetentionPolicy enforces a minimum retention time for all objects
contained in the bucket.
Any attempt to overwrite or delete objects younger than the retention
period will result in an error. An unlocked retention policy can be
modified or removed from the bucket via the Update method. A
locked retention policy cannot be removed or shortened in duration
for the lifetime of the bucket.
This feature is in private alpha release. It is not currently available to
most customers. It might be changed in backwards-incompatible ways and is not
subject to any SLA or deprecation policy.
EffectiveTime is the time from which the policy was enforced and
effective. This field is read-only.
IsLocked describes whether the bucket is locked. Once locked, an
object retention policy cannot be modified.
This field is read-only.
RetentionPeriod specifies the duration that objects need to be
retained. Retention duration must be greater than zero and less than
100 years. Note that enforcement of retention periods less than a day
is not guaranteed. Such periods should only be used for testing
purposes.
(*T) toRawRetentionPolicy() *raw.BucketRetentionPolicy
func toRetentionPolicy(rp *raw.BucketRetentionPolicy) (*RetentionPolicy, error)
SignedURLOptions allows you to restrict the access to the signed URL.
ContentType is the content type header the client must provide
to use the generated signed URL.
Optional.
Expires is the expiration time on the signed URL. It must be
a datetime in the future. For SigningSchemeV4, the expiration may be no
more than seven days in the future.
Required.
GoogleAccessID represents the authorizer of the signed URL generation.
It is typically the Google service account client email address from
the Google Developers Console in the form of "xxx@developer.gserviceaccount.com".
Required.
Headers is a list of extension headers the client must provide
in order to use the generated signed URL. Each must be a string of the
form "key:values", with multiple values separated by a semicolon.
Optional.
Insecure determines whether the signed URL should use HTTPS (default) or
HTTP.
Only supported for V4 signing.
Optional.
MD5 is the base64 encoded MD5 checksum of the file.
If provided, the client should provide the exact value on the request
header in order to use the signed URL.
Optional.
Method is the HTTP method to be used with the signed URL.
Signed URLs can be used with GET, HEAD, PUT, and DELETE requests.
Required.
PrivateKey is the Google service account private key. It is obtainable
from the Google Developers Console.
At https://console.developers.google.com/project/<your-project-id>/apiui/credential,
create a service account client ID or reuse one of your existing service account
credentials. Click on the "Generate new P12 key" to generate and download
a new private key. Once you download the P12 file, use the following command
to convert it into a PEM file.
$ openssl pkcs12 -in key.p12 -passin pass:notasecret -out key.pem -nodes
Provide the contents of the PEM file as a byte slice.
Exactly one of PrivateKey or SignBytes must be non-nil.
QueryParameters is a map of additional query parameters. When
SigningScheme is V4, this is used in computing the signature, and the
client must use the same query parameters when using the generated signed
URL.
Optional.
Scheme determines the version of URL signing to use. Default is
SigningSchemeV2.
SignBytes is a function for implementing custom signing. For example, if
your application is running on Google App Engine, you can use
appengine's internal signing function:
ctx := appengine.NewContext(request)
acc, _ := appengine.ServiceAccount(ctx)
url, err := SignedURL("bucket", "object", &SignedURLOptions{
GoogleAccessID: acc,
SignBytes: func(b []byte) ([]byte, error) {
_, signedBytes, err := appengine.SignBytes(ctx, b)
return signedBytes, err
},
// etc.
})
Exactly one of PrivateKey or SignBytes must be non-nil.
Style provides options for the type of URL to use. Options are
PathStyle (default), BucketBoundHostname, and VirtualHostedStyle. See
https://cloud.google.com/storage/docs/request-endpoints for details.
Only supported for V4 signing.
Optional.
func SignedURL(bucket, name string, opts *SignedURLOptions) (string, error)
func signedURLV2(bucket, name string, opts *SignedURLOptions) (string, error)
func signedURLV4(bucket, name string, opts *SignedURLOptions, now time.Time) (string, error)
func validateOptions(opts *SignedURLOptions, now time.Time) error
UniformBucketLevelAccess configures access checks to use only bucket-level IAM
policies.
Enabled specifies whether access checks use only bucket-level IAM
policies. Enabled may be disabled until the locked time.
LockedTime specifies the deadline for changing Enabled from true to
false.
func toUniformBucketLevelAccess(b *raw.BucketIamConfiguration) UniformBucketLevelAccess
URLStyle determines the style to use for the signed URL. pathStyle is the
default. All non-default options work with V4 scheme only. See
https://cloud.google.com/storage/docs/request-endpoints for details.
host should return the host portion of the signed URL, not including
the scheme (e.g. storage.googleapis.com).
path should return the path portion of the signed URL, which may include
both the bucket and object name or only the object name depending on the
style.
bucketBoundHostnamepathStylevirtualHostedStyle
func BucketBoundHostname(hostname string) URLStyle
func PathStyle() URLStyle
func VirtualHostedStyle() URLStyle
A Writer writes a Cloud Storage object.
ChunkSize controls the maximum number of bytes of the object that the
Writer will attempt to send to the server in a single request. Objects
smaller than the size will be sent in a single request, while larger
objects will be split over multiple requests. The size will be rounded up
to the nearest multiple of 256K.
ChunkSize will default to a reasonable value. If you perform many
concurrent writes of small objects (under ~8MB), you may wish set ChunkSize
to a value that matches your objects' sizes to avoid consuming large
amounts of memory. See
https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload#size
for more information about performance trade-offs related to ChunkSize.
If ChunkSize is set to zero, chunking will be disabled and the object will
be uploaded in a single request without the use of a buffer. This will
further reduce memory used during uploads, but will also prevent the writer
from retrying in case of a transient error from the server, since a buffer
is required in order to retry the failed request.
ChunkSize must be set before the first Write call.
ObjectAttrs are optional attributes to set on the object. Any attributes
must be initialized before the first Write call. Nil or zero-valued
attributes are ignored.
ACL is the list of access control rules for the object.
Bucket is the name of the bucket containing this GCS object.
This field is read-only.
CRC32C is the CRC32 checksum of the object's content using the Castagnoli93
polynomial. This field is read-only, except when used from a Writer or
Composer. In those cases, if the SendCRC32C field in the Writer or Composer
is set to is true, the uploaded data is rejected if its CRC32C hash does
not match this field.
CacheControl is the Cache-Control header to be sent in the response
headers when serving the object data.
ContentDisposition is the optional Content-Disposition header of the object
sent in the response headers.
ContentEncoding is the encoding of the object's content.
ContentLanguage is the content language of the object's content.
ContentType is the MIME type of the object's content.
Created is the time the object was created. This field is read-only.
CustomerKeySHA256 is the base64-encoded SHA-256 hash of the
customer-supplied encryption key for the object. It is empty if there is
no customer-supplied encryption key.
See // https://cloud.google.com/storage/docs/encryption for more about
encryption in Google Cloud Storage.
Deleted is the time the object was deleted.
If not deleted, it is the zero value. This field is read-only.
Etag is the HTTP/1.1 Entity tag for the object.
This field is read-only.
EventBasedHold specifies whether an object is under event-based hold. New
objects created in a bucket whose DefaultEventBasedHold is set will
default to that value.
Generation is the generation number of the object's content.
This field is read-only.
Cloud KMS key name, in the form
projects/P/locations/L/keyRings/R/cryptoKeys/K, used to encrypt this object,
if the object is encrypted by such a key.
Providing both a KMSKeyName and a customer-supplied encryption key (via
ObjectHandle.Key) will result in an error when writing an object.
MD5 is the MD5 hash of the object's content. This field is read-only,
except when used from a Writer. If set on a Writer, the uploaded
data is rejected if its MD5 hash does not match this field.
MediaLink is an URL to the object's content. This field is read-only.
Metadata represents user-provided metadata, in key/value pairs.
It can be nil if no metadata is provided.
Metageneration is the version of the metadata for this
object at this generation. This field is used for preconditions
and for detecting changes in metadata. A metageneration number
is only meaningful in the context of a particular generation
of a particular object. This field is read-only.
Name is the name of the object within the bucket.
This field is read-only.
Owner is the owner of the object. This field is read-only.
If non-zero, it is in the form of "user-<userId>".
If not empty, applies a predefined set of access controls. It should be set
only when writing, copying or composing an object. When copying or composing,
it acts as the destinationPredefinedAcl parameter.
PredefinedACL is always empty for ObjectAttrs returned from the service.
See https://cloud.google.com/storage/docs/json_api/v1/objects/insert
for valid values.
Prefix is set only for ObjectAttrs which represent synthetic "directory
entries" when iterating over buckets using Query.Delimiter. See
ObjectIterator.Next. When set, no other fields in ObjectAttrs will be
populated.
RetentionExpirationTime is a server-determined value that specifies the
earliest time that the object's retention period expires.
This is a read-only field.
Size is the length of the object's content. This field is read-only.
StorageClass is the storage class of the object. This defines
how objects are stored and determines the SLA and the cost of storage.
Typical values are "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
Defaults to "STANDARD".
See https://cloud.google.com/storage/docs/storage-classes for all
valid values.
TemporaryHold specifies whether an object is under temporary hold. While
this flag is set to true, the object is protected against deletion and
overwrites.
Updated is the creation or modification time of the object.
For buckets with versioning enabled, changing an object's
metadata does not change this property. This field is read-only.
ProgressFunc can be used to monitor the progress of a large write.
operation. If ProgressFunc is not nil and writing requires multiple
calls to the underlying service (see
https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload),
then ProgressFunc will be invoked after each call with the number of bytes of
content copied so far.
ProgressFunc should return quickly without blocking.
SendCRC specifies whether to transmit a CRC32C field. It should be set
to true in addition to setting the Writer's CRC32C field, because zero
is a valid CRC and normally a zero would not be transmitted.
If a CRC32C is sent, and the data written does not match the checksum,
the write will be rejected.
ctxcontext.Context
// closed after err and obj are set.
errerrormusync.Mutexo*ObjectHandleobj*ObjectAttrsopenedboolpw*io.PipeWriter
Attrs returns metadata about a successfully-written object.
It's only valid to call it after Close returns nil.
Close completes the write operation and flushes any buffered data.
If Close doesn't return an error, metadata about the written object
can be retrieved by calling Attrs.
CloseWithError aborts the write operation with the provided error.
CloseWithError always returns nil.
Deprecated: cancel the context passed to NewWriter instead.
Write appends to w. It implements the io.Writer interface.
Since writes happen asynchronously, Write may return a nil
error even though the write failed (or will fail). Always
use the error returned from Writer.Close to determine if
the upload was successful.
Writes will be retried on transient errors from the server, unless
Writer.ChunkSize has been set to zero.
monitorCancel is intended to be used as a background goroutine. It monitors the
context, and when it observes that the context has been canceled, it manually
closes things that do not take a context.
(*T) open() error
toRawObject copies the editable attributes from o to the raw library's Object type.
*T : github.com/go-git/go-git/v5/plumbing/protocol/packp/sideband.Progress
*T : github.com/jbenet/go-context/io.Writer
*T : io.Closer
*T : io.WriteCloser
*T : io.Writer
func (*ObjectHandle).NewWriter(ctx context.Context) *Writer
Package-Level Functions (total 71, in which 11 are exported)
BucketBoundHostname generates a URL with a custom hostname tied to a
specific GCS bucket. The desired hostname should be passed in using the
hostname argument. Generated urls will be of the form
"<bucket-bound-hostname>/<object-name>". See
https://cloud.google.com/storage/docs/request-endpoints#cname and
https://cloud.google.com/load-balancing/docs/https/adding-backend-buckets-to-load-balancers
for details. Note that for CNAMEs, only HTTP is supported, so Insecure must
be set to true.
ConditionContentLengthRange constraints the limits that the
multipart upload's range header will be expected to be within.
ConditionStartsWith checks that an attributes starts with value.
An empty value will cause this condition to be ignored.
ForHMACKeyServiceAccountEmail returns HMAC Keys that are
associated with the email address of a service account in the project.
Only one service account email can be used as a filter, so if multiple
of these options are applied, the last email to be set will be used.
This option is EXPERIMENTAL and subject to change or removal without notice.
GenerateSignedPostPolicyV4 generates a PostPolicyV4 value from bucket, object and opts.
The generated URL and fields will then allow an unauthenticated client to perform multipart uploads.
NewClient creates a new Google Cloud Storage client.
The default scope is ScopeFullControl. To use a different scope, like ScopeReadOnly, use option.WithScopes.
PathStyle is the default style, and will generate a URL of the form
"storage.googleapis.com/<bucket-name>/<object-name>".
ShowDeletedHMACKeys will also list keys whose state is "DELETED".
This option is EXPERIMENTAL and subject to change or removal without notice.
SignedURL returns a URL for the specified object. Signed URLs allow
the users access to a restricted resource for a limited time without having a
Google account or signing in. For more information about the signed
URLs, see https://cloud.google.com/storage/docs/accesscontrol#Signed-URLs.
UserProjectForHMACKeys will bill the request against userProjectID
if userProjectID is non-empty.
Note: This is a noop right now and only provided for API compatibility.
This option is EXPERIMENTAL and subject to change or removal without notice.
VirtualHostedStyle generates a URL relative to the bucket's virtual
hostname, e.g. "<bucket-name>.storage.googleapis.com/<object-name>".
applyBucketConds modifies the provided call using the conditions in conds.
call is something that quacks like a *raw.WhateverCall.
applyConds modifies the provided call using the conditions in conds.
call is something that quacks like a *raw.WhateverCall.
conditionsQuery returns the generation and conditions as a URL query
string suitable for URL.RawQuery. It assumes that the conditions
have been validated.
convertTime converts a time in RFC3339 format to time.Time.
If any error occurs in parsing, the zero-value time.Time is silently returned.
Decode a uint32 encoded in Base64 in big-endian byte order.
decompressiveTranscoding returns true if the request was served decompressed
and different than its original storage form. This happens when the "Content-Encoding"
header is "gzip".
See:
* https://cloud.google.com/storage/docs/transcoding#transcoding_and_gzip
* https://github.com/googleapis/google-cloud-go/issues/1800
Encode a uint32 as Base64 in big-endian byte order.
extractHeaderNames takes in a series of key:value headers and returns the
header names only.
parseKey converts the binary contents of a private key file to an
*rsa.PrivateKey. It detects whether the private key is in a PEM container or
not. If so, it extracts the private key from PEM container before
conversion. It only supports PEM containers with no passphrase.
parseNotificationTopic extracts the project and topic IDs from from the full
resource name returned by the service. If the name is malformed, it returns
"?" for both IDs.
pathEncodeV4 creates an encoded string that matches the v4 signature spec.
Following the spec precisely is necessary in order to ensure that the URL
and signing string are correctly formed, and Go's url.PathEncode and
url.QueryEncode don't generate an exact match without some additional logic.
setConditionField sets a field on a *raw.WhateverCall.
We can't use anonymous interfaces because the return type is
different, since the field setters are builders.
v2SanitizeHeaders applies the specifications for canonical extension headers at
https://cloud.google.com/storage/docs/access-control/signed-urls#about-canonical-extension-headers.
v4SanitizeHeaders applies the specifications for canonical extension headers
at https://cloud.google.com/storage/docs/access-control/signed-urls#about-canonical-extension-headers.
V4 does a couple things differently from V2:
- Headers get sorted by key, instead of by key:value. We do this in
signedURLV4.
- There's no canonical regexp: we simply split headers on :.
- We don't exclude canonical headers.
- We replace leading and trailing spaces in header values, like v2, but also
all intermediate space duplicates get stripped. That is, there's only ever
a single consecutive space.
validateMetadata ensures that all keys passed in have a prefix of "x-goog-meta-",
otherwise it will return an error.
validatePostPolicyV4Options checks that:
* GoogleAccessID is set
* either but not both PrivateKey and SignBytes are set or nil, but not both
* Expires, the deadline is not in the past
* if Style is not set, it'll use PathStyle
Package-Level Variables (total 15, in which 2 are exported)
ErrBucketNotExist indicates that the bucket does not exist.
ErrObjectNotExist indicates that the object does not exist.
attrToFieldMap maps the field names of ObjectAttrs to the underlying field
names in the API call. Only the ObjectAttrs field names are visible to users
because they are already part of the public API of the package.
DeleteAction is a lifecycle action that deletes a live and/or archived
objects. Takes precedence over SetStorageClass actions.
Deleted is the status for a key that is deleted.
Once in this state the key cannot key cannot be recovered
and does not count towards key limits. Deleted keys will be cleaned
up later.
Inactive is the status for an inactive key thus requests signed by
this key will be denied.
Send object metadata as JSON with notification messages.
Live specifies that the object is still live.
LiveAndArchived includes both live and archived objects.
Send no payload with notification messages.
Event that occurs when the live version of an object becomes an
archived version.
Event that occurs when an object is permanently deleted.
Event that occurs when an object is successfully created.
Event that occurs when the metadata of an existing object changes.
The pages are generated with Goldsv0.3.2-preview. (GOOS=darwin GOARCH=amd64)
Golds is a Go 101 project developed by Tapir Liu.
PR and bug reports are welcome and can be submitted to the issue list.
Please follow @Go100and1 (reachable from the left QR code) to get the latest news of Golds.