Copyright 2019 The Go Authors. All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file.
Package zip provides functions for creating and extracting module zip files. Module zip files have several restrictions listed below. These are necessary to ensure that module zip files can be extracted consistently on supported platforms and file systems. • All file paths within a zip file must start with "<module>@<version>/", where "<module>" is the module path and "<version>" is the version. The module path must be valid (see golang.org/x/mod/module.CheckPath). The version must be valid and canonical (see golang.org/x/mod/module.CanonicalVersion). The path must have a major version suffix consistent with the version (see golang.org/x/mod/module.Check). The part of the file path after the "<module>@<version>/" prefix must be valid (see golang.org/x/mod/module.CheckFilePath). • No two file paths may be equal under Unicode case-folding (see strings.EqualFold). • A go.mod file may or may not appear in the top-level directory. If present, it must be named "go.mod", not any other case. Files named "go.mod" are not allowed in any other directory. • The total size in bytes of a module zip file may be at most MaxZipFile bytes (500 MiB). The total uncompressed size of the files within the zip may also be at most MaxZipFile bytes. • Each file's uncompressed size must match its declared 64-bit uncompressed size in the zip file header. • If the zip contains files named "<module>@<version>/go.mod" or "<module>@<version>/LICENSE", their sizes in bytes may be at most MaxGoMod or MaxLICENSE, respectively (both are 16 MiB). • Empty directories are ignored. File permissions and timestamps are also ignored. • Symbolic links and other irregular files are not allowed. Note that this package does not provide hashing functionality. See golang.org/x/mod/sumdb/dirhash.
package zip

import (
	
	
	
	
	
	
	
	
	
	
	
	

	
)

MaxZipFile is the maximum size in bytes of a module zip file. The go command will report an error if either the zip file or its extracted content is larger than this.
	MaxZipFile = 500 << 20
MaxGoMod is the maximum size in bytes of a go.mod file within a module zip file.
	MaxGoMod = 16 << 20
MaxLICENSE is the maximum size in bytes of a LICENSE file within a module zip file.
	MaxLICENSE = 16 << 20
)
File provides an abstraction for a file in a directory, zip, or anything else that looks like a file.
Path returns a clean slash-separated relative path from the module root directory to the file.
	Path() string
Lstat returns information about the file. If the file is a symbolic link, Lstat returns information about the link itself, not the file it points to.
	Lstat() (os.FileInfo, error)
Open provides access to the data within a regular file. Open may return an error if called on a directory or symbolic link.
	Open() (io.ReadCloser, error)
}
CheckedFiles reports whether a set of files satisfy the name and size constraints required by module zip files. The constraints are listed in the package documentation. Functions that produce this report may include slightly different sets of files. See documentation for CheckFiles, CheckDir, and CheckZip for details.
Valid is a list of file paths that should be included in a zip file.
Omitted is a list of files that are ignored when creating a module zip file, along with the reason each file is ignored.
Invalid is a list of files that should not be included in a module zip file, along with the reason each file is invalid.
SizeError is non-nil if the total uncompressed size of the valid files exceeds the module zip size limit or if the zip file itself exceeds the limit.
Err returns an error if CheckedFiles does not describe a valid module zip file. SizeError is returned if that field is set. A FileErrorList is returned if there are one or more invalid files. Other errors may be returned in the future.
func ( CheckedFiles) () error {
	if .SizeError != nil {
		return .SizeError
	}
	if len(.Invalid) > 0 {
		return FileErrorList(.Invalid)
	}
	return nil
}

type FileErrorList []FileError

func ( FileErrorList) () string {
	 := &strings.Builder{}
	 := ""
	for ,  := range  {
		.WriteString()
		.WriteString(.Error())
		 = "\n"
	}
	return .String()
}

type FileError struct {
	Path string
	Err  error
}

func ( FileError) () string {
	return fmt.Sprintf("%s: %s", .Path, .Err)
}

func ( FileError) () error {
	return .Err
}

Predefined error messages for invalid files. Not exhaustive.
	errPathNotClean    = errors.New("file path is not clean")
	errPathNotRelative = errors.New("file path is not relative")
	errGoModCase       = errors.New("go.mod files must have lowercase names")
	errGoModSize       = fmt.Errorf("go.mod file too large (max size is %d bytes)", MaxGoMod)
	errLICENSESize     = fmt.Errorf("LICENSE file too large (max size is %d bytes)", MaxLICENSE)
Predefined error messages for omitted files. Not exhaustive.
	errVCS           = errors.New("directory is a version control repository")
	errVendored      = errors.New("file is in vendor directory")
	errSubmoduleFile = errors.New("file is in another module")
	errSubmoduleDir  = errors.New("directory is in another module")
	errHgArchivalTxt = errors.New("file is inserted by 'hg archive' and is always omitted")
	errSymlink       = errors.New("file is a symbolic link")
	errNotRegular    = errors.New("not a regular file")
)
CheckFiles reports whether a list of files satisfy the name and size constraints listed in the package documentation. The returned CheckedFiles record contains lists of valid, invalid, and omitted files. Every file in the given list will be included in exactly one of those lists. CheckFiles returns an error if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when an error is returned. Note that CheckFiles will not open any files, so Create may still fail when CheckFiles is successful due to I/O errors and reported size differences.
func ( []File) (CheckedFiles, error) {
	, ,  := checkFiles()
	return , .Err()
}
checkFiles implements CheckFiles and also returns lists of valid files and their sizes, corresponding to cf.Valid. These lists are used in Crewate to avoid repeated calls to File.Lstat.
func ( []File) ( CheckedFiles,  []File,  []int64) {
	 := make(map[string]struct{})
	 := func( string,  bool,  error) {
		if ,  := [];  {
			return
		}
		[] = struct{}{}
		 := FileError{Path: , Err: }
		if  {
			.Omitted = append(.Omitted, )
		} else {
			.Invalid = append(.Invalid, )
		}
	}
Find directories containing go.mod files (other than the root). Files in these directories will be omitted. These directories will not be included in the output zip.
	 := make(map[string]bool)
	for ,  := range  {
		 := .Path()
		,  := path.Split()
		if strings.EqualFold(, "go.mod") {
			,  := .Lstat()
			if  != nil {
				(, false, )
				continue
			}
			if .Mode().IsRegular() {
				[] = true
			}
		}
	}

	 := func( string) bool {
		for {
			,  := path.Split()
			if  == "" {
				return false
			}
			if [] {
				return true
			}
			 = [:len()-1]
		}
	}

	 := make(collisionChecker)
	 := int64(MaxZipFile)
	for ,  := range  {
		 := .Path()
		if  != path.Clean() {
			(, false, errPathNotClean)
			continue
		}
		if path.IsAbs() {
			(, false, errPathNotRelative)
			continue
		}
		if isVendoredPackage() {
			(, true, errVendored)
			continue
		}
		if () {
			(, true, errSubmoduleFile)
			continue
		}
Inserted by hg archive. The go command drops this regardless of the VCS being used.
			(, true, errHgArchivalTxt)
			continue
		}
		if  := module.CheckFilePath();  != nil {
			(, false, )
			continue
		}
		if strings.ToLower() == "go.mod" &&  != "go.mod" {
			(, false, errGoModCase)
			continue
		}
		,  := .Lstat()
		if  != nil {
			(, false, )
			continue
		}
		if  := .check(, .IsDir());  != nil {
			(, false, )
			continue
		}
Skip symbolic links (golang.org/issue/27093).
			(, true, errSymlink)
			continue
		}
		if !.Mode().IsRegular() {
			(, true, errNotRegular)
			continue
		}
		 := .Size()
		if  >= 0 &&  <=  {
			 -= 
		} else if .SizeError == nil {
			.SizeError = fmt.Errorf("module source tree too large (max size is %d bytes)", MaxZipFile)
		}
		if  == "go.mod" &&  > MaxGoMod {
			(, false, errGoModSize)
			continue
		}
		if  == "LICENSE" &&  > MaxLICENSE {
			(, false, errLICENSESize)
			continue
		}

		.Valid = append(.Valid, )
		 = append(, )
		 = append(, .Size())
	}

	return , , 
}
CheckDir reports whether the files in dir satisfy the name and size constraints listed in the package documentation. The returned CheckedFiles record contains lists of valid, invalid, and omitted files. If a directory is omitted (for example, a nested module or vendor directory), it will appear in the omitted list, but its files won't be listed. CheckDir returns an error if it encounters an I/O error or if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when such an error is returned. Note that CheckDir will not open any files, so CreateFromDir may still fail when CheckDir is successful due to I/O errors.
List files (as CreateFromDir would) and check which ones are omitted or invalid.
	, ,  := listFilesInDir()
	if  != nil {
		return CheckedFiles{}, 
	}
	,  := CheckFiles()
	_ =  // ignore this error; we'll generate our own after rewriting paths.
Replace all paths with file system paths. Paths returned by CheckFiles will be slash-separated paths relative to dir. That's probably not appropriate for error messages.
	for  := range .Valid {
		.Valid[] = filepath.Join(, .Valid[])
	}
	.Omitted = append(.Omitted, ...)
	for  := range .Omitted {
		.Omitted[].Path = filepath.Join(, .Omitted[].Path)
	}
	for  := range .Invalid {
		.Invalid[].Path = filepath.Join(, .Invalid[].Path)
	}
	return , .Err()
}
CheckZip reports whether the files contained in a zip file satisfy the name and size constraints listed in the package documentation. CheckZip returns an error if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when an error is returned. CheckZip will also return an error if the module path or version is malformed or if it encounters an error reading the zip file. Note that CheckZip does not read individual files, so Unzip may still fail when CheckZip is successful due to I/O errors.
func ( module.Version,  string) (CheckedFiles, error) {
	,  := os.Open()
	if  != nil {
		return CheckedFiles{}, 
	}
	defer .Close()
	, ,  := checkZip(, )
	return , 
}
checkZip implements checkZip and also returns the *zip.Reader. This is used in Unzip to avoid redundant I/O.
Make sure the module path and version are valid.
	if  := module.CanonicalVersion(.Version);  != .Version {
		return nil, CheckedFiles{}, fmt.Errorf("version %q is not canonical (should be %q)", .Version, )
	}
	if  := module.Check(.Path, .Version);  != nil {
		return nil, CheckedFiles{}, 
	}
Check the total file size.
	,  := .Stat()
	if  != nil {
		return nil, CheckedFiles{}, 
	}
	 := .Size()
	if  > MaxZipFile {
		 := CheckedFiles{SizeError: fmt.Errorf("module zip file is too large (%d bytes; limit is %d bytes)", , MaxZipFile)}
		return nil, , .Err()
	}
Check for valid file names, collisions.
	var  CheckedFiles
	 := func( *zip.File,  error) {
		.Invalid = append(.Invalid, FileError{Path: .Name, Err: })
	}
	,  := zip.NewReader(, )
	if  != nil {
		return nil, CheckedFiles{}, 
	}
	 := fmt.Sprintf("%s@%s/", .Path, .Version)
	 := make(collisionChecker)
	var  int64
	for ,  := range .File {
		if !strings.HasPrefix(.Name, ) {
			(, fmt.Errorf("path does not have prefix %q", ))
			continue
		}
		 := .Name[len():]
		if  == "" {
			continue
		}
		 := strings.HasSuffix(, "/")
		if  {
			 = [:len()-1]
		}
		if path.Clean() !=  {
			(, errPathNotClean)
			continue
		}
		if  := module.CheckFilePath();  != nil {
			(, )
			continue
		}
		if  := .check(, );  != nil {
			(, )
			continue
		}
		if  {
			continue
		}
		if  := path.Base(); strings.EqualFold(, "go.mod") {
			if  !=  {
				(, fmt.Errorf("go.mod file not in module root directory"))
				continue
			}
			if  != "go.mod" {
				(, errGoModCase)
				continue
			}
		}
		 := int64(.UncompressedSize64)
		if  >= 0 && MaxZipFile- >=  {
			 += 
		} else if .SizeError == nil {
			.SizeError = fmt.Errorf("total uncompressed size of module contents too large (max size is %d bytes)", MaxZipFile)
		}
		if  == "go.mod" &&  > MaxGoMod {
			(, fmt.Errorf("go.mod file too large (max size is %d bytes)", MaxGoMod))
			continue
		}
		if  == "LICENSE" &&  > MaxLICENSE {
			(, fmt.Errorf("LICENSE file too large (max size is %d bytes)", MaxLICENSE))
			continue
		}
		.Valid = append(.Valid, .Name)
	}

	return , , .Err()
}
Create builds a zip archive for module m from an abstract list of files and writes it to w. Create verifies the restrictions described in the package documentation and should not produce an archive that Unzip cannot extract. Create does not include files in the output archive if they don't belong in the module zip. In particular, Create will not include files in modules found in subdirectories, most files in vendor directories, or irregular files (such as symbolic links) in the output archive.
func ( io.Writer,  module.Version,  []File) ( error) {
	defer func() {
		if  != nil {
			 = &zipError{verb: "create zip", err: }
		}
	}()
Check that the version is canonical, the module path is well-formed, and the major version suffix matches the major version.
	if  := module.CanonicalVersion(.Version);  != .Version {
		return fmt.Errorf("version %q is not canonical (should be %q)", .Version, )
	}
	if  := module.Check(.Path, .Version);  != nil {
		return 
	}
Check whether files are valid, not valid, or should be omitted. Also check that the valid files don't exceed the maximum size.
	, ,  := checkFiles()
	if  := .Err();  != nil {
		return 
	}
Create the module zip file.
	 := zip.NewWriter()
	 := fmt.Sprintf("%s@%s/", .Path, .Version)

	 := func( File,  string,  int64) error {
		,  := .Open()
		if  != nil {
			return 
		}
		defer .Close()
		,  := .Create( + )
		if  != nil {
			return 
		}
		 := &io.LimitedReader{R: , N:  + 1}
		if ,  := io.Copy(, );  != nil {
			return 
		}
		if .N <= 0 {
			return fmt.Errorf("file %q is larger than declared size", )
		}
		return nil
	}

	for ,  := range  {
		 := .Path()
		 := []
		if  := (, , );  != nil {
			return 
		}
	}

	return .Close()
}
CreateFromDir creates a module zip file for module m from the contents of a directory, dir. The zip content is written to w. CreateFromDir verifies the restrictions described in the package documentation and should not produce an archive that Unzip cannot extract. CreateFromDir does not include files in the output archive if they don't belong in the module zip. In particular, CreateFromDir will not include files in modules found in subdirectories, most files in vendor directories, or irregular files (such as symbolic links) in the output archive. Additionally, unlike Create, CreateFromDir will not include directories named ".bzr", ".git", ".hg", or ".svn".
func ( io.Writer,  module.Version,  string) ( error) {
	defer func() {
		if ,  := .(*zipError);  {
			.path = 
		} else if  != nil {
			 = &zipError{verb: "create zip", path: , err: }
		}
	}()

	, ,  := listFilesInDir()
	if  != nil {
		return 
	}

	return Create(, , )
}

type dirFile struct {
	filePath, slashPath string
	info                os.FileInfo
}

func ( dirFile) () string                 { return .slashPath }
func ( dirFile) () (os.FileInfo, error)  { return .info, nil }
func ( dirFile) () (io.ReadCloser, error) { return os.Open(.filePath) }
isVendoredPackage attempts to report whether the given filename is contained in a package whose import path contains (but does not end with) the component "vendor". Unfortunately, isVendoredPackage reports false positives for files in any non-top-level package whose import path ends in "vendor".
func ( string) bool {
	var  int
	if strings.HasPrefix(, "vendor/") {
		 += len("vendor/")
This offset looks incorrect; this should probably be i = j + len("/vendor/") (See https://golang.org/issue/31562 and https://golang.org/issue/37397.) Unfortunately, we can't fix it without invalidating module checksums.
		 += len("/vendor/")
	} else {
		return false
	}
	return strings.Contains([:], "/")
}
Unzip extracts the contents of a module zip file to a directory. Unzip checks all restrictions listed in the package documentation and returns an error if the zip archive is not valid. In some cases, files may be written to dir before an error is returned (for example, if a file's uncompressed size does not match its declared size). dir may or may not exist: Unzip will create it and any missing parent directories if it doesn't exist. If dir exists, it must be empty.
func ( string,  module.Version,  string) ( error) {
	defer func() {
		if  != nil {
			 = &zipError{verb: "unzip", path: , err: }
		}
	}()
Check that the directory is empty. Don't create it yet in case there's an error reading the zip.
	if ,  := ioutil.ReadDir(); len() > 0 {
		return fmt.Errorf("target directory %v exists and is not empty", )
	}
Open the zip and check that it satisfies all restrictions.
	,  := os.Open()
	if  != nil {
		return 
	}
	defer .Close()
	, ,  := checkZip(, )
	if  != nil {
		return 
	}
	if  := .Err();  != nil {
		return 
	}
Unzip, enforcing sizes declared in the zip file.
	 := fmt.Sprintf("%s@%s/", .Path, .Version)
	if  := os.MkdirAll(, 0777);  != nil {
		return 
	}
	for ,  := range .File {
		 := .Name[len():]
		if  == "" || strings.HasSuffix(, "/") {
			continue
		}
		 := filepath.Join(, )
		if  := os.MkdirAll(filepath.Dir(), 0777);  != nil {
			return 
		}
		,  := os.OpenFile(, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0444)
		if  != nil {
			return 
		}
		,  := .Open()
		if  != nil {
			.Close()
			return 
		}
		 := &io.LimitedReader{R: , N: int64(.UncompressedSize64) + 1}
		_,  = io.Copy(, )
		.Close()
		if  != nil {
			.Close()
			return 
		}
		if  := .Close();  != nil {
			return 
		}
		if .N <= 0 {
			return fmt.Errorf("uncompressed size of file %s is larger than declared size (%d bytes)", .Name, .UncompressedSize64)
		}
	}

	return nil
}
collisionChecker finds case-insensitive name collisions and paths that are listed as both files and directories. The keys of this map are processed with strToFold. pathInfo has the original path for each folded path.
type collisionChecker map[string]pathInfo

type pathInfo struct {
	path  string
	isDir bool
}

func ( collisionChecker) ( string,  bool) error {
	 := strToFold()
	if ,  := [];  {
		if  != .path {
			return fmt.Errorf("case-insensitive file name collision: %q and %q", .path, )
		}
		if  != .isDir {
			return fmt.Errorf("entry %q is both a file and a directory", )
		}
		if ! {
			return fmt.Errorf("multiple entries for file %q", )
It's not an error if check is called with the same directory multiple times. check is called recursively on parent directories, so check may be called on the same directory many times.
	} else {
		[] = pathInfo{path: , isDir: }
	}

	if  := path.Dir();  != "." {
		return .(, true)
	}
	return nil
}
listFilesInDir walks the directory tree rooted at dir and returns a list of files, as well as a list of directories and files that were skipped (for example, nested modules and symbolic links).
func ( string) ( []File,  []FileError,  error) {
	 = filepath.Walk(, func( string,  os.FileInfo,  error) error {
		if  != nil {
			return 
		}
		,  := filepath.Rel(, )
		if  != nil {
			return 
		}
		 := filepath.ToSlash()
Skip some subdirectories inside vendor, but maintain bug golang.org/issue/31562, described in isVendoredPackage. We would like Create and CreateFromDir to produce the same result for a set of files, whether expressed as a directory tree or zip.
		if isVendoredPackage() {
			 = append(, FileError{Path: , Err: errVendored})
			return nil
		}

		if .IsDir() {
Don't skip the top-level directory.
				return nil
			}
Skip VCS directories. fossil repos are regular files with arbitrary names, so we don't try to exclude them.
			switch filepath.Base() {
			case ".bzr", ".git", ".hg", ".svn":
				 = append(, FileError{Path: , Err: errVCS})
				return filepath.SkipDir
			}
Skip submodules (directories containing go.mod files).
			if ,  := os.Lstat(filepath.Join(, "go.mod"));  == nil && !.IsDir() {
				 = append(, FileError{Path: , Err: errSubmoduleDir})
				return filepath.SkipDir
			}
			return nil
		}
Skip irregular files and files in vendor directories. Irregular files are ignored. They're typically symbolic links.
		if !.Mode().IsRegular() {
			 = append(, FileError{Path: , Err: errNotRegular})
			return nil
		}

		 = append(, dirFile{
			filePath:  ,
			slashPath: ,
			info:      ,
		})
		return nil
	})
	if  != nil {
		return nil, nil, 
	}
	return , , nil
}

type zipError struct {
	verb, path string
	err        error
}

func ( *zipError) () string {
	if .path == "" {
		return fmt.Sprintf("%s: %v", .verb, .err)
	} else {
		return fmt.Sprintf("%s %s: %v", .verb, .path, .err)
	}
}

func ( *zipError) () error {
	return .err
}
strToFold returns a string with the property that strings.EqualFold(s, t) iff strToFold(s) == strToFold(t) This lets us test a large set of strings for fold-equivalent duplicates without making a quadratic number of calls to EqualFold. Note that strings.ToUpper and strings.ToLower do not have the desired property in some corner cases.
Fast path: all ASCII, no upper case. Most paths look like this already.
	for  := 0;  < len(); ++ {
		 := []
		if  >= utf8.RuneSelf || 'A' <=  &&  <= 'Z' {
			goto 
		}
	}
	return 

:
	var  bytes.Buffer
SimpleFold(x) cycles to the next equivalent rune > x or wraps around to smaller values. Iterate until it wraps, and we've found the minimum value.
		for {
			 := 
			 = unicode.SimpleFold()
			if  <=  {
				break
			}
Exception to allow fast path above: A-Z => a-z
		if 'A' <=  &&  <= 'Z' {
			 += 'a' - 'A'
		}
		.WriteRune()
	}
	return .String()