package postgres

Import Path
	golang.org/x/pkgsite/internal/postgres (on go.dev)

Dependency Relation
	imports 58 packages, and imported by 5 packages



Package-Level Type Names (total 13, in which 7 are exported)


	/* sort exporteds by:  |  */
	
		ASTTransformer is a default transformer of the goldmark tree.

		
			
				Transform transforms the given AST tree to remove an unnecessary child
				node from the image node. This is so that the summary generated doesn't
				the text content of an image block.

		
			*T : github.com/yuin/goldmark/parser.ASTTransformer


	

		
			
		
			
				CleanModule deletes all versions of the given module path from the DB and marks them
				as cleaned in module_version_states.

			
				CleanModuleVersions deletes each module version from the DB and marks it as cleaned
				in module_version_states.

			
				Close closes a DB.

			
				DeleteModule deletes a Version from the database.

			
				DeleteOlderVersionFromSearchDocuments deletes from search_documents every package with
				the given module path whose version is older than the given version.
				It is used when fetching a module with an alternative path. See internal/worker/fetch.go:fetchAndUpdateState.

			
				DeletePseudoversionsExcept deletes all pseudoversions for the module except
				the provided resolvedVersion.

			
				GetExcludedPrefixes reads all the excluded prefixes from the database.

			
				GetImportedBy fetches and returns all of the packages that import the
				package with path.
				The returned error may be checked with derrors.IsInvalidArgument to
				determine if it resulted from an invalid package path or version.

				Instead of supporting pagination, this query runs with a limit.

			
				GetImportedByCount returns the number of packages that import pkgPath.

			
				GetLatestInfo returns the latest information about the unit in the module.
				See internal.LatestInfo for documentation about the returned values.
				If latestUnitMeta is non-nil, it is the result of GetUnitMeta(unitPath, internal.UnknownModulePath, internal.LatestVersion).
				That can save a redundant call to GetUnitMeta here.

			
				GetLatestMajorPathForV1Path reports the latest unit path in the series for
				the given v1path. It also returns the major version for that path.

			
				GetLatestModuleVersions returns the row of the latest_module_versions table for modulePath.
				If the module path is not found, it returns nil, nil.

			
				GetModuleInfo fetches a module version from the database with the primary key
				(module_path, version).

			
				GetModuleReadme returns the README corresponding to the modulePath and version.

			
				GetModuleVersionState returns the current module version state for
				modulePath and version.

			
				GetModuleVersionsToClean returns module versions that can be removed from the database.
				Only module versions that were updated more than daysOld days ago will be considered.
				At most limit module versions will be returned.

			
				GetNestedModules returns the latest major version of all nested modules
				given a modulePath path prefix with or without major version.

			
				GetNextModulesToFetch returns the next batch of modules that need to be
				processed. We prioritize modules based on (1) whether it has status zero
				(never processed), (2) whether it is the latest version, (3) if it is an
				alternative module, and (4) the number of packages it has. We want to leave
				time-consuming modules until the end and process them at a slower rate to
				reduce database load and timeouts. We also want to leave alternative modules
				towards the end, since these will incur unnecessary deletes otherwise.

			
				GetPackageVersionState returns the current package version state for
				pkgPath, modulePath and version.

			
				GetPackageVersionStatesForModule returns the current package version states
				for modulePath and version.

			
				GetPackagesForSearchDocumentUpsert fetches search information for packages in search_documents
				whose update time is before the given time.

			
				GetRecentFailedVersions returns versions that have most recently failed.

			
				GetRecentVersions returns recent versions that have been processed.

			
				GetStdlibPathsWithSuffix returns information about all paths in the latest version of the standard
				library whose last component is suffix. A path that exactly match suffix is not included;
				the path must end with "/" + suffix.

				We are only interested in actual standard library packages: not commands, which we happen to include
				in the stdlib module, and not directories (paths that do not contain a package).

			
				GetSymbolHistory returns a SymbolHistory, which is a representation of the
				first version when a symbol is added to an API.

			
				GetUnit returns a unit from the database, along with all of the data
				associated with that unit.
				If bc is not nil, get only the Documentation that matches it (or nil if none do).

			
				GetUnitMeta returns information about the "best" entity (module, path or directory) with
				the given path. The module and version arguments provide additional constraints.
				If the module is unknown, pass internal.UnknownModulePath; if the version is unknown, pass
				internal.LatestVersion.

				The rules for picking the best are:
				1. If the version is known but the module path is not, choose the longest module path
				   at that version that contains fullPath.
				2. Otherwise, find the latest "good" version (in the modules table) that contains fullPath.
				   a. First, follow the algorithm of the go command: prefer longer module paths, and
				      find the latest unretracted version, using semver but preferring release to pre-release.
				   b. If no modules have latest-version information, find the latest by sorting the versions
				      we do have: again first by module path length, then by version.

			
				GetVersionMap fetches a version_map entry corresponding to the given
				modulePath and requestedVersion.

			
				GetVersionMaps returns all of the version maps for the provided
				path and requested version if they are present.

			
				GetVersionStats queries the module_version_states table for aggregate
				information about the current state of module versions, grouping them by
				their current status code.

			
				GetVersionsForPath returns a list of tagged versions sorted in
				descending semver order if any exist. If none, it returns the 10 most
				recent from a list of pseudo-versions sorted in descending semver order.

			
				HasGoMod reports whether a given module version has a go.mod file.
				It returns a NotFound error if it can't find any information.

			
				InsertExcludedPrefix inserts prefix into the excluded_prefixes table.

				For real-time administration (e.g. DOS prevention), use the dbadmin tool.
				to exclude or unexclude a prefix. If the exclusion is permanent (e.g. a user
				request), also add the prefix and reason to the excluded.txt file.

			
				InsertIndexVersions inserts new versions into the module_version_states
				table with a status of zero.

			
				InsertModule inserts a version into the database using db.saveVersion, along
				with a search document corresponding to each of its packages.
				It returns whether the version inserted was the latest for the given module path.

			
				IsExcluded reports whether the path matches the excluded list.
				A path matches an entry on the excluded list if it equals the entry, or
				is a component-wise suffix of the entry.
				So path "bad/ness" matches entries "bad" and "bad/", but path "badness"
				matches neither of those.

			
				LatestIndexTimestamp returns the last timestamp successfully inserted into
				the module_version_states table.

			
				ReInsertLatestVersion checks that the latest good version matches the version
				in search_documents. If it doesn't, it inserts the latest good version into
				search_documents and imports_unique.

			
				Search executes two search requests concurrently:
				  - a sequential scan of packages in descending order of popularity.
				  - all packages ("deep" search) using an inverted index to filter to search
				    terms.

				The sequential scan takes significantly less time when searching for very
				common terms (e.g. "errors", "cloud", or "kubernetes"), due to its ability
				to exit early once the requested page of search results is provably
				complete.

				Because 0 <= ts_rank() <= 1, we know that the highest score of any unscanned
				package is ln(e+N), where N is imported_by_count of the package we are
				currently considering.  Therefore if the lowest scoring result of popular
				search is greater than ln(e+N), we know that we haven't missed any results
				and can return the search result immediately, cancelling other searches.

				On the other hand, if the popular search is slow, it is likely that the
				search term is infrequent, and deep search will be fast due to our inverted
				gin index on search tokens.

				The gap in this optimization is search terms that are very frequent, but
				rarely relevant: "int" or "package", for example. In these cases we'll pay
				the penalty of a deep search that scans nearly every package.

			
				StalenessTimestamp returns the index timestamp of the oldest
				module that is newer than the index timestamp of the youngest module we have
				processed. That is, let T be the maximum index timestamp of all processed
				modules. Then this function return the minimum index timestamp of unprocessed
				modules that is no less than T, or an error that wraps derrors.NotFound if
				there is none.

				The name of the function is imprecise: there may be an older unprocessed
				module, if one newer than it has been processed.

				We use this function to compute a metric that is a lower bound on the time
				it takes to process a module since it appeared in the index.

			
				Underlying returns the *database.DB inside db.

			
				UpdateLatestModuleVersions upserts its argument into the latest_module_versions table
				if the row doesn't exist, or the new version is later.
				It returns the version that is in the DB when it completes.

			
				UpdateLatestModuleVersionsStatus updates or inserts a failure status into the
				latest_module_versions table.
				It only updates the table if it doesn't have valid information for the module path.

			
				UpdateModuleVersionStatesForReprocessing marks modules to be reprocessed
				that were processed prior to the provided appVersion.

			
				UpdateModuleVersionStatesForReprocessingLatestOnly marks modules to be
				reprocessed that were processed prior to the provided appVersion.

			
				UpdateModuleVersionStatesForReprocessingReleaseVersionsOnly marks modules to be
				reprocessed that were processed prior to the provided appVersion.

			(*T) UpdateModuleVersionStatesWithStatus(ctx context.Context, status int, appVersion string) (err error)
			
				UpdateModuleVersionStatus updates the status and error fields of a module version.

			
				UpdateSearchDocumentsImportedByCount updates imported_by_count and
				imported_by_count_updated_at.

				It does so by completely recalculating the imported-by counts
				from the imports_unique table.

				UpdateSearchDocumentsImportedByCount returns the number of rows updated.

			
				UpsertModuleVersionState inserts or updates the module_version_state table with
				the results of a fetch operation for a given module version.

			
				UpsertSearchDocumentWithImportedByCount is the same as UpsertSearchDocument,
				except it also updates the imported by count. This is only used for testing.

			
				UpsertVersionMap inserts a version_map entry into the database.

			
		
			*T : golang.org/x/pkgsite/internal.DataSource
			*T : io.Closer
		
			func New(db *database.DB) *DB
			func NewBypassingLicenseCheck(db *database.DB) *DB
			func SetupTestDB(dbName string) (_ *DB, err error)
			func golang.org/x/pkgsite/cmd/internal/cmdconfig.OpenDB(ctx context.Context, cfg *config.Config, bypassLicenseCheck bool) (_ *DB, err error)
			
		
			func GetFromSearchDocuments(ctx context.Context, t *testing.T, db *DB, packagePath string) (modulePath, version string, found bool)
			func InsertSampleDirectoryTree(ctx context.Context, t *testing.T, testDB *DB)
			func MustInsertModule(ctx context.Context, t *testing.T, db *DB, m *internal.Module)
			func MustInsertModuleGoMod(ctx context.Context, t *testing.T, db *DB, m *internal.Module, goMod string)
			func MustInsertModuleNotLatest(ctx context.Context, t *testing.T, db *DB, m *internal.Module)
			func ResetTestDB(db *DB, t *testing.T)
			func RunDBTests(dbName string, m *testing.M, testDB **DB)
			func golang.org/x/pkgsite/internal/frontend.FetchAndUpdateState(ctx context.Context, modulePath, requestedVersion string, proxyClient *proxy.Client, sourceClient *source.Client, db *DB) (_ int, err error)
			


	
		HTMLRenderer is a renderer.NodeRenderer implementation that renders
		pkg.go.dev readme features.

		
			Config html.Config
			Config.HardWraps bool
			Config.Unsafe bool
			Config.Writer html.Writer
			Config.XHTML bool
		
			
				RegisterFuncs implements renderer.NodeRenderer.RegisterFuncs.

			
				SetOption implements renderer.NodeRenderer.SetOption.

			
		
			*T : github.com/yuin/goldmark/renderer.NodeRenderer
			*T : github.com/yuin/goldmark/renderer.SetOptioner


	
		A ModuleVersion holds a module path and version.

		
			ModulePath string
			Version string
		
			( T) String() string
		
			 T : expvar.Var
			 T : fmt.Stringer
			
		
			func (*DB).GetModuleVersionsToClean(ctx context.Context, daysOld, limit int) (modvers []ModuleVersion, err error)
		
			func (*DB).CleanModuleVersions(ctx context.Context, mvs []ModuleVersion, reason string) (err error)


	

		
			AppVersion string
			FetchErr error
			GoModPath string
			HasGoMod bool
			ModulePath string
			PackageVersionStates []*internal.PackageVersionState
			Status int
			Timestamp time.Time
			Version string
		
			func (*DB).UpsertModuleVersionState(ctx context.Context, mvs *ModuleVersionStateForUpsert) (err error)
			


	

		
			ModulePath string
			PackagePath string
			ReadmeContents string
			ReadmeFilePath string
			Synopsis string
			Version string
		
			func (*DB).GetPackagesForSearchDocumentUpsert(ctx context.Context, before time.Time, limit int) (argsList []UpsertSearchDocumentArgs, err error)
		
			func UpsertSearchDocument(ctx context.Context, ddb *database.DB, args UpsertSearchDocumentArgs) (err error)
			func (*DB).UpsertSearchDocumentWithImportedByCount(ctx context.Context, args UpsertSearchDocumentArgs, importedByCount int) (err error)


	
		VersionStats holds statistics extracted from the module_version_states
		table.

		
			LatestTimestamp time.Time
			
				// from status to number of rows

		
			func (*DB).GetVersionStats(ctx context.Context) (_ *VersionStats, err error)


	


Package-Level Functions (total 93, in which 19 are exported)


	
		GeneratePathTokens returns the subPaths and path token parts that will be
		indexed for search, which includes (1) the packagePath (2) all sub-paths of
		the packagePath (3) all parts for a path element that is delimited by a dash
		and (4) all parts of a path element that is delimited by a dot, except for
		the last element.


	
		GetFromSearchDocuments retrieves the module path and version for the given
		package path from the search_documents table. If the path is not in the table,
		the third return value is false.


	 func GetPathID(ctx context.Context, ddb *database.DB, path string) (id int, err error)
	
		GetSymbolHistoryForBuildContext returns a map of the first version when a symbol name is
		added to the API for the specified build context, to the symbol name, to the
		UnitSymbol struct. The UnitSymbol.Children field will always be empty, as
		children names are also tracked.


	
		GetSymbolHistoryFromTable returns a SymbolHistory, which is a representation of the
		first version when a symbol is added to an API. It reads data from the
		symbol_history table.


	
		GetSymbolHistoryWithPackageSymbols fetches symbol history data by using data
		from package_symbols and documentation_symbols, and computed using
		symbol.IntroducedHistory.

		GetSymbolHistoryWithPackageSymbols is exported for use in tests.


	
		InsertSampleDirectory tree inserts a set of packages for testing
		GetUnit and frontend.FetchDirectoryDetails.


	
		MustInsertModule inserts m into db, calling t.Fatal on error.
		It also updates the latest-version information for m.


	
		New returns a new postgres DB.


	
		NewBypassingLicenseCheck returns a new postgres DB that bypasses license
		checks. That means all data will be inserted and returned for
		non-redistributable modules, packages and directories.


	
		NewHTMLRenderer creates a new HTMLRenderer for a readme.


	
		ResetTestDB truncates all data from the given test DB.  It should be called
		after every test that mutates the database.


	
		RunDBTests is a wrapper that runs the given testing suite in a test database
		named dbName.  The given *DB reference will be set to the instantiated test
		database.


	
		RunDBTestsInParallel sets up numDBs databases, then runs the tests. Before it runs them,
		it sets acquirep to a function that tests should use to acquire a database. The second
		return value of the function should be called in a defer statement to release the database.
		For example:

		   func Test(t *testing.T) {
		       db, release := acquire(t)
		       defer release()


	
		SearchDocumentSections computes the B and C sections of a Postgres search
		document from a package synopsis and a README.
		By "B section" and "C section" we mean the portion of the tsvector with weight
		"B" and "C", respectively.

		The B section consists of the synopsis.
		The C section consists of the first sentence of the README.
		The D section consists of the remainder of the README.
		All sections are split into words and processed for replacements.
		Each section is limited to maxSectionWords words, and in addition the
		D section is limited to an initial fraction of the README, determined
		by maxReadmeFraction.


	
		SetupTestDB creates a test database named dbName if it does not already
		exist, and migrates it to the latest schema from the migrations directory.


	
		UpsertSearchDocument inserts a row in search_documents for the given package.
		The given module should have already been validated via a call to
		validateModule.


	


Package-Level Variables (total 15, in which 2 are exported)


	
		SearchLatencyDistribution aggregates search request latency by search
		query type.


	
		SearchResponseCount counts search responses by search query type.


	


Package-Level Constants (total 8, none are exported)