Documentation

What is a package?

A package is a combination of metadata, configuration, and software that is prepared in a way that a package management program (for example: apt on Ubuntu, yum on Redhat Enterprise Linux, pip for Python packages and gem for RubyGems) can use to properly and reliably install software and related configuration data on a computer.

Why are packages useful?

• Version information helps keep software up to date.
• Metadata offers visibility in to what's installed where and why.
• Reproducibility: software is installed the same way, everywhere.

How do I create packages?

There are many tools for creating packages. Some tools are provided directly by Linux distributions. There are also many great third-party tools.

Some popular tools that can be used to create packages:

• rpmbuild for RPM packages. Also, take a look at this tutorial.
• debuild (includes an excellent tutorial) for DEB packages.
• distutils for Python packages.
• gem (includes an excellent tutorial) for RubyGems.

Advanced and third-party tools:

• mock a chroot-based system for building RPM packages in a clean room environment.
• pbuilder a chroot-based system for building DEB packages in a clean room environment. This page also includes some useful tips about pbuilder.
• git-buildpackage a set of scripts that can be used to build DEB packages directly from git repositories.
• fpm a third-party tool that allows users to quickly and easily make a variety of packages (including RPM and DEB packages).
• packpack a simple tool to build RPM and Debian packages from git repositories.

Which types of packages does packagecloud.io support?

Currently packagecloud.io supports:

• RPM packages
• DEB packages
• Debian source packages (DSCs)
• Java packages (Clojure, SBT, "fatjar")
• Python packages (wheels, eggs, source distributions)
• RubyGems
• Node.js packages

We are actively working on support for additional package formats. Email us and let us know if we're missing something you'd like to see.

What is a repo?

A repo (also known as 'repository') is a collection of packages and some metadata describing a variety of attributes about the packages (for example: package versions, operating system version, dependencies, processor architecture, etc).

What is a packagecloud repo?

A packagecloud repo is a more advanced take on the traditional repo.

A single packagecloud repo can:

• Contain packages of any (or all!) supported types; for example Debian, RPM, RubyGem, and Python packages can all coexist in the same packagecloud repo.
• Have packages for multiple Linux distributions; for example, if you have a Debian package that works for 2 versions of Ubuntu and 1 version of Debian you only need 1 packagecloud repo (see pushing packages for more info).
• Issue read tokens to identify specific nodes and control access to a repo by specific node.

Why are repos useful?

Repos are useful because they:

• Allow you store several versions of the same package (useful for debugging purposes, security analysis, etc).
• Typically include some verification mechanism (like GPG).
• Offer a way for many remote machines to download and install the same packages.
• Provide a way for software vendors to offer stable releases of software to customers.

How do I create repos?

There are a few open source tools that you can use to create a repo. Some of the mostly commonly used tools are:

• createrepo for creating and managing RPM package repos (also called YUM repos).
• reprepro for creating and managing DEB package repos (also called APT repos).
• gem for creating and managing RubyGem servers.

Unfortunately, many of these tools are buggy, poorly documented, or difficult to use in production.

An excellent alternative is packagecloud.io ;) - you can read more about how to create repos with our command line tool here. You can also create repos on packagecloud.io by logging in and clicking the New Repository button or by clicking here.

Disabling filelists metadata

APT and YUM repositories have metadata which contains a list of every file in every package. This metadata can be used by users to query a given a filename and determine which package wrote that file to the filesystem.

The major downside of this feature is that repositories with huge numbers of packages or packages with huge numbers of files (like Omnibus packages) will have extremely large file list metadata. This can cause long wait times for clients because their apt and yum clients will download this metadata to the system when refreshing the cached metadata for a repository. This excessive metadata download can also increase the length of configuration management runs and take up significant space on the user's disk.

In order to deal with this, repository owners can click the 'edit' link to the right of their repository on their homepage and check the checkbox which says "Disable filelist metadata". Save your changes by clicking update. This will trigger a re-index for your repository and your contents metadata will be empty.

Debian repository component support

Since we make it easy to create and manage as many repositories as you want, we feel there is no real need to support Debian repository components.

GPG keys

A GPG key can be used to generate a digital signature (read more about how GPG generates and verifies digital signatures).

Many package manage systems that come with Linux support two different types of GPG signatures. GPG signatures on packages themselves and GPG signatures on repository metadata. The distincition is very subtle, but very important.

GPG signed packages

If you want to GPG sign your packages, you should do so before uploading it to packagecloud. packagecloud does not modify your uploaded data in anyway, whatsoever. If you sign your packages before uploading them, they will be signed when downloaded.

packagecloud allows repository owners to upload the GPG key used to sign the packages in a repository. When you do this, the key URL will be included in the package manager config that is generated for the client host. This ensures the key will be downloaded, added to the keyring, and used to verify package signatures. Note that packagecloud only stores public keys. If you happen to upload a secret key, we'll discard the secret key data and store only the public key when we process your GPG key upload.

GPG signed repository metadata

packagecloud does GPG sign repository metadata that we generate for your repository. We do this to ensure to our users that your repository metadata was generated by us.

If you've installed your packagecloud repository using the Bash script, Chef cookbook, or Puppet module there is nothing extra you need to do. All of those methods automatically install the correct GPG key for your repository.

Adding and removing GPG keys

For more information about adding or removing the GPG keys for a repository, visit the repository's page and click GPG. Read the next section to learn about how to export a GPG key that you've used to sign your packages so it can be uploaded to packagecloud.

Exporting GPG keys

To export a GPG key that you've used to sign packages uploaded to packagecloud, follow these instructions.

1. Begin by running gpg --list-keys --keyid-format=long. This will print information about each GPG key in your keyring. It will look like this:

   pub   2048R/F2BB309992AE12C3 2015-03-14
   uid                          Fake Name (test key) 
   sub   2048R/5F042FCD59BF0D31 2015-03-14
                   

2. Find the key ID for the package signing key you are using. In the example above, the key ID is F2BB309992AE12C3.
3. Export the public key by running: gpg --armor --export keyid, substituting keyid for your GPG key ID.
4. Copy and paste this output into the package key dialog located in the Package signing key section of the GPG tab on your repository.

Learn more

To learn more about the technical inner workings of GPG signing packages vs repository metadata, check out our blog posts about GPG signing and verifying Debian packages and APT repositories and GPG signing and verifying RPM packages and YUM repositories.

GPG key migration

Some users have received notifications that they must migrate their repositories to a new GPG key. This does not affect all repositories; only repositories created prior to our new GPG key system being rolled out are affected.

Legacy GPG keys

In the early days of packagecloud, the service used a global GPG key to sign the repository metadata generated for all APT and YUM repositories created.

This method works, but is a bit inflexible. Soon after we migrated to a new system which uses AWS Key Management Service to generate GPG signing keys per-repository.

Per-repository GPG keys

Whenever a repository is created on packagecloud, a new GPG key is generated specifically for that repository to be used for signing repository metadata.

We are now migrating all repositories that were created before this system was rolled out over to this new system. Any repositories created after are fine as is and no action is required.

How to migrate

The migration can be performed in stages and we encourage affected users to click the GPG tab on their repository page to more information on how to perform the migration.

Any repository that is not migrated will be automatically migrated on January 7, 2019.

Feel free to reach out to us with any questions about this process: support@packagecloud.io

HTTPS

Both public and private repos have their metadata and package files served up over HTTPS to prevent MITM attacks.

packagecloud repo install scripts explicitly turn on certificate verification when possible.

Public / Private Repos

packagecloud allows you to create as many public repos as you like. Depending on your plan you may also be able to create private repos, as well.

• Read accessible by anyone.
• Best suited for companies and individuals who are intending to distribute open source software.
• The repo owner and the specified collaborators may push packages to the repo.

• Read accessible only with a read token
• Best suited for companies and individuals who have proprietary software.
• The repo owner and the specified collaborators may push packages to the repo.

Collaborators

packagecloud allows you to authorize other packagecloud users to push packages to and yank packages from your repos, as well as promote packages to other repositories.

You can add or remove collaborators by clicking the edit link found next to a repo name when you log in to packagecloud.

Pushing and installing rpm_any packages

About rpm_any packages

Rpm packages, uploaded as rpm_any, can be created to work on any rpm distros. This allows for a single package for every version, which can serve for all rpm-based Linuxes.

Pushing a package using the CLI:

You can push a deb any/any package using the packagecloud CLI, like so:

package_cloud push <username>/<reponame>/rpm_any/rpm_any <rpm file>

Using the rpm_any/rpm_any repo for installation:

1. Create a repo definition pointing to the rpm_any/rpm_any repo on packagecloud, and store it in /etc/yum.repos.d/. The filename MUST have the extension .repo. Example:

   /etc/yum.repos.d/username_reponame_rpm_any.repo

   The file content should be:

       [username_reponame_rpm_any]
       name=username_reponame_rpm_any
       baseurl=https://packagecloud.io/username/reponame/rpm_any/rpm_any/$basearch
       repo_gpgcheck=1
       gpgcheck=0
       enabled=1
       gpgkey=https://packagecloud.io/username/reponame/gpgkey
       sslverify=1
       sslcacert=/etc/pki/tls/certs/ca-bundle.crt
       metadata_expire=300

2. Update and get the metadata for the rpm_any/rpm_any repo on packagecloud. This command below will pull in the packages available on the repo pointed to by username_reponame_rpm_any, and you will be prompted to accept the GPG key of the repo.

   yum update --disablerepo=* --enablerepo=username_reponame_rpm_any

3. Confirm all available packages from the repo you added by running:

   yum list | grep username_reponame_rpm_any

4. Install the desired package with yum install or the below command, which will ensure you will install the package only from the repo pointed to by username_reponame_rpm_any

   yum install <package name> --disablerepo=* --enablerepo=username_reponame_rpm_any

Questions? We're here to help! Email support@packagecloud.io

Pushing and installing deb any/any packages

About deb any/any packages

Deb packages, uploaded as deb any/any, can be created to work on any deb distros. This allows for a single package for every version, which can serve for all deb-based Linuxes.

Pushing a package using the CLI:

You can push a deb any/any package using the packagecloud CLI, like so:

package_cloud push <username>/<reponame>/any/any <deb file>

Using the any/any repo for installation:

1. Create a repo definition pointing to the any/any repo on packagecloud, and store it in /etc/apt/sources.list.d/. The filename MUST have the extension .list. Example:

   /etc/apt/sources.list.d/username_reponame_deb_any.list

   The file content should be:

       deb https://packagecloud.io/username/reponame/any/ any main
       deb-src https://packagecloud.io/username/reponame/any/ any main

2. Update and get the metadata for the any/any repo on packagecloud. This command will pull in the packages available on the repo pointed to by username_reponame_deb_any.

       sudo apt-get update \
       -o Dir::Etc::sourcelist="sources.list.d/username_reponame_deb_any.list" \
       -o Dir::Etc::sourceparts="-" \
       -o APT::Get::List-Cleanup="0"

3. Confirm the metadata from the repo you added by running:

   ls /var/apt/lists/

   The content should display the following:

   packagecloud.io_username_repo_any_dists_any_InRelease

4. Review which repo the package is coming from:

   apt-get install --print-uris <package name>

5. Install the desired package:

   apt-get install <package name>

Questions? We're here to help! Email support@packagecloud.io

Bundler

Adding the packagecloud repository to your Gemfile

Bundler 1.7.0 and above

In newer bundler versions, you can scope specific gems to a source, like so:

public repos:

For public repositories, you'll only need to add it as a source to your Gemfile.

# Gemfile
#
# Note: It's recommended you add the official https://rubygems.org source, unless your
#       packagecloud repository can meet all of the dependency requirements in the Gemfile.

source "https://rubygems.org"
source "https://packagecloud.io/user/publicRepo" do
  gem "my-gem"
  gem "another-gem"
end

private repos:

To install gems from a private repository, you'll need to add it as a source to your Gemfile with a read token. See generating read tokens.

# Gemfile
# Replace ${token} with a valid read token

source "https://rubygems.org"
source "https://${token}:@packagecloud.io/user/privateRepo" do
  gem "my-private-gem"
  gem "another-private-gem"
end

Legacy versions of Bundler

Older versions of Bundler have several bugs around scoping gems to a single source using blocks, so you'll have to add the source globally at the top of the Gemfile.

public repos:

# Gemfile

source "https://rubygems.org"
source "https://packagecloud.io/user/publicRepo"

private repos:

# Gemfile
# Replace ${token} with a valid read token.

source "https://rubygems.org"
source "https://${token}:@packagecloud.io/user/privateRepo"

VirtualEnv

Adding the repository to your VirtualEnv

Ensure you are running the latest version of pip inside your virtualenv:

my_virtualenv/bin/pip install --upgrade pip

public repos:

Add this to the bottom of your requirements.txt

--extra-index-url=https://packagecloud.io/user/publicRepo/pypi/simple

private repos:

Append the repository to requirements.txt using a read token. Replace ${token} with your generated read token. See generating read tokens.

--extra-index-url=https://${token}:@packagecloud.io/user/privateRepo/pypi/simple

Note: if you would like pip to use only this repository as a source, replace extra-index-url with index-url

Configure an NPM registry

Use the npm config set registry <url> command to set the NPM registry for your system.

This command will create an .npmrc file with the custom registry url in the current user's home directory. The url should be a fully-qualified url to a packagecloud repository and must include the trailing npm/.

For example, to set the NPM registry to example-user's repository named example-repo, you would run the following command:

npm config set registry https://packagecloud.io/example-user/example-repo/npm/

Follow the directions in the next sections to set up read-only access or read and write access to the repository.

Read only access to NPM repositories

Public NPM repositories

If the repostory is a public repository, your system will have read access to the repository after running npm config set registry <url> as described in the previous section.

You can install packages by simply running npm install [packagename].

Private NPM repositories

To configure read only access to a private NPM registry, use the repository installation scripts which can be found in the Installation section on a repository page.

Alternatively, you can create a read token and and set the _authToken manually by editing the .npmrc file in your user's home directory.

always-auth=true
registry=https://packagecloud.io/example-user/example-repo/npm/
//packagecloud.io/example-user/example-repo/npm/:_authToken=f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0

The _authToken must be a valid read token generated from a repository master token. Learn more about read tokens in these docs.

The always-auth field is required when using Yarn with a private repository.

Read and write access to NPM repositories

You should only follow these instructions on systems and environments where you need write access to your repository. If you just need to run npm install to install packages, you should follow the read only access guide above.

npm login

After you have set your repository URL by following the guide above, you are ready to use npm login to give yourself write access to the repository.

Start by using the npm login command. Running this command will modify the .npmrc file in your home directory to include an authentication token that has write access.

When running the npm login command, you will be prompted for a username, password, and email address. Use your packagecloud account name, your packagecloud account password, and packagecloud account email address, respectively.

For example, if your packagecloud username is example-user and the email address registered to your account is example-user@company.com, you would run npm login and respond like this:

~/Projects/example-project $ npm login
Username: example-user
Password: **************
Email: (this IS public) example-user@company.com
Logged in as example-user on https://packagecloud.io/example-user/example-repo/npm/.

npm publish

After you have followed the npm login section above, you are ready to use npm publish.

Use the npm cli to publish packages to your packagecloud registry. Run npm publish <tarball> or npm publish from inside a project root containing a package.json to publish packages to a configured registry.

You can also use the packagecloud CLI, API, or web interface to upload packages to the NPM registry.

Scoped packages

Scoped packages are supported on packagecloud. Install scoped packages by including the scope in the package name.

npm install @example/package

Publishing scoped packages is also supported. Use the npm publish command, packagecloud CLI, or web UI to upload packages. Ensure the scope is included in the package name field inside the package.json.

npm publish example-package-v0.1.0.tgz

Remove / unpublish packages from the registry

To remove packages from your packagecloud registry, you can use the packagecloud CLI or the API directly to delete the package.

CLI Example:

package_cloud yank example-user/example-repository/node example-1.0.tgz

To remove scoped packages from your packagecloud registry, you pass the scope to the CLI, like so:

package_cloud yank example-user/example-repository/node @myuser/example-1.0.tgz

You can also delete packages from the web UI by visiting the repository page, selecting the package you want to delete, and choosing delete from the options presented. Deleting the package will trigger a reindex of the registry metadata.

Transparent auto-proxying

When you install a package using npm install, the npm program will automatically attempt to install any dependencies required by the package. If those dependencies are not found in your packagecloud repository, packagecloud will automatically forward requests for those missing dependencies to the official NPM public registry.

This is the default behavior and is recommended for most users.

If you do not want to rely on the official NPM public registry at all, you can disable the automatic forwarding of requests. However, once this behavior is disabled, you will need to upload your packages and all required dependencies to your packagecloud repository. To disable automatic proxying to the official NPM public registry for missing dependencies, visit the settings page for the repository and uncheck the "Enable proxying from the official NPM registry" checkbox.

If you decide to disable this behavior, you will have to upload every dependency for every package you upload. npm install commands will fail if any dependencies are missing from your packagecloud repository because npm install will no longer be able to rely on the official NPM public registry.

Using Yarn

Once a system is configured to use a packagecloud repository as an NPM registry, either by following the manual installation instructions, or by using the repository install scripts provided by packagecloud, it can install packages via Yarn by using yarn add <packagename>. No other configuration is necessary.

yarn add packagename@0.1.0

NPM dist tags

NPM supports a useful feature called distribution tags (or dist tags) for short. You can learn about dist tags by reading the NPM documentation.

Please read the important notes about dist tags in the sections that follow. The behavior of dist tags with package publish, promote, and delete may not be what you expect!

In short, NPM dist tags allow NPM package owners to associate arbitrary strings with NPM package versions. All packages have at least one dist tag: the "latest" dist tag. The "latest" dist tag is automatically set to the most recently uploaded version of a package. When you run npm install package, your npm command line client automatically installs the version tagged with "latest".

You can create arbitrary dist tags (for example: alpha, beta, testing, stable, etc) that map to particular versions of a package. A user can then use that dist tag to install the version of a package mapped to that tag.

For example: you may decide that version 2.0 of example-package is the current beta version, so you create a dist tag mapping "beta" to version 2.0. Any user who wants to install the beta, can simply run npm install example-package@beta.

The NPM API for adding, removing, and listing dist tags is fully supported by packagecloud. This means you can run commands like: npm dist-tag ls package-name or npm dist-tag add package-name@2.0 beta to list or add dist tags, respectively.

Publishing a package and distribution tags (dist tags)

In order to maintain API compatibility with the official NPM registry and its API you should be be aware that when uploading a Node.js package with the:

• Web-based graphical upload dialog, the package uploaded will be marked as the latest version regardless of its version string.
• package_cloud command line client, the package uploaded will be marked as the latest version regardless of its version string.
• npm publish command without specifying a distribution tag, the newly published package will be marked as the latest version regardless of its version string.
• npm publish command and also specifying a distribution tag using the --tag option, the newly published package will be marked with the specified dist tag. The "latest" distribution tag will remain unchanged and will be set to whichever version it was set to (if any).

You can add, remove, or list dist tags using the npm dist-tags command to modify the dist tags, manually.

Node package promotion and distribution tags (dist tags)

When you promote a package, the package is removed from the source repository and exists only in the destination repository at the completion of the command.

When a Node.js package is promoted, the follow things occur:

1. All distribution tags associated with that package are deleted.
2. The remaining versions of the package in the original repository (where the package was moved from) will be sorted according to the version. The most recent version string is marked as the latest (even if it already has another dist tag).
3. Once the package reaches the destination repository, it will be marked as latest if (and only if) no other versions of the same package exist at the destination. If other versions exist, the version that was just moved has no dist tag associated with it, even if the semantic version is the latest.

This is done to mimic the official NPM repository as closely as possible. You can use the npm dist-tags command to create, remove, or adjust any dist tags you need after promoting a package.

Node package deletion and distribution tags (dist tags)

When you delete (or yank) a Node.js package the version strings of all remaining versions for that package are sorted and the most recent version is marked as "latest" even if there is already another dist tag associated with that version. This is done to maintain API compatibility with the official NPM registry and its API.

Token Capabilities

* This includes downloading packages made available through repository metadata.

** For compatibility with the npm tool, API tokens can read and write to NPM registries.

Master Tokens

Master tokens are at the top level and their only purpose is generating descendent read tokens.

Master tokens can't be used for reading from repos, pushing packages, modifying configurations, or doing anything else. That means that you can safely give master tokens to customers, embed them in configuration management manifests, or otherwise distribute them to untrusted parties.

Each repo comes with a default master token. If you don't have a need for maintaining fine-grained access control to your repos, then all packagecloud functionality will automatically use the default master token transparently to you.

Master tokens can be generated and destroyed with the package_cloud command line client. For example, to create a new token named test, you would run the following command:

> package_cloud master_token create username/reponame master_token_name

Read Tokens

Read tokens are what's used to authenticate against the packagecloud repository. In other words, the package manager on your system will send this token along with its requests for repo metadata and packages.

These tokens are read-only, meaning that they can't be used to modify the repo in any way or authenticate against any packagecloud endpoints aside from the package servers themselves.We recommend using one read token per node to maximize isolation.

Read tokens are generated automatically by our repo installation scripts. Our scripts automatically associate read tokens with a node's hostname. If you write your own config management manifests, generating and associating read tokens is a simple matter of making a call to a REST API.

Revocation

If you want to revoke access to one node at a time, you can revoke its read token using the CLI. This is one of the benefits of generating one read-token per node.

> package_cloud read_token destroy username/reponame master_token_name/read_token_name

By revoking a master token, you can revoke all of its read token descendents at once, making it straightforward to — at once — revoke an entire customer, datacenter, or other logical grouping that exists in your system.

> package_cloud master_token destroy username/reponame master_token_name

Generating a Read Token for Private Repo Access

Generating a read token is a matter of making a REST call to the tokens endpoint for your repository. The following example will generate and return a read token for the private repository.

The ${master_token} in the example is referring to the master token associated to a repository. See Master Tokens.

NOTE: The installation scripts provided by packagecloud automatically generate read tokens for private repositories when they're executed. Take a look at the installation instructions for any packagecloud repository.

UNIQUE_ID=`hostname -f` && curl -XPOST --data "name=${UNIQUE_ID}" https://${master_token}:@packagecloud.io/install/repositories/${username}/${repository}/tokens.text

To push packages to packagecloud from CircleCI you must:

1. Set an environment variable named PACKAGECLOUD_TOKEN in your project's settings and the value must match your packagecloud API access token.
2. Create a circle.yml file which installs the package_cloud gem and pushes the package to the OS and version of your choice.

For more info on your API access token, please refer to the API Tokens section in API docs.

Questions? We're here to help! Email support@packagecloud.io

PACKAGECLOUD_TOKEN: f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0

dependencies:
  pre:
    - gem install package_cloud

deployment:
  production:
    branch: master
    commands:
      - package_cloud push user/repo/os/version /path/to/pkg.ext

To push packages to packagecloud from Travis CI you must:

1. Set an environment variable named PACKAGECLOUD_TOKEN in your project's settings and the value must match your packagecloud API access token.
2. Create a .travis.yml file which installs the package_cloud gem and pushes the package to the OS and version of your choice.

For more info on your API access token, please refer to the API Tokens section in API docs.

Questions? We're here to help! Email support@packagecloud.io

PACKAGECLOUD_TOKEN: f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0

before_install:
  - gem install package_cloud

script:
  - package_cloud push user/repo/os/version /path/to/package.ext

To push packages to packagecloud from Jenkins you'll need to install and configure the packagecloud Jenkins plugin.

The step by step instructions can be found on the packagecloud Jenkins plugin wiki page.

There are 3 steps to the configuration process:

1. Plugin installation: begin by finding and installing the Jenkins plugin via the Jenkins UI
2. Credentials setup: next, use Jenkins' credentials system to store your packagecloud username and API token
3. Configure post-build job: finally, add a post-build step to your Jenkins job, enter the repository to push to, and select the operating system and version

You'll need to use your packagecloud username and API token when configuring credentials in Jenkins.

Please log in to packagecloud to view your username and API token.

Questions? We're here to help! Email support@packagecloud.io

To push packages to packagecloud from Buildkite you must:

1. Ensure the package_cloud gem is installed in your agent environment
2. Expose the PACKAGECLOUD_TOKEN environment variable in your Buildkite environment hook
3. Add a build script to push your build artifact to packagecloud

For more info on your API access token, please refer to the API Tokens section in API docs.

Questions? We're here to help! Email support@packagecloud.io

#!/bin/bash

export PACKAGECLOUD_TOKEN=f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0

#!/bin/bash

set -e

if [[ $# -eq 0 ]]; then
  echo "Usage: $0 username/myrepo path/to/package/artifact"
  exit 1
fi

# Download package artifact from a previous build step
buildkite-agent artifact download "$1" .

# Push package to packagecloud
package_cloud push "$2" "$(basename $1)"
  

NOTE: This GitHub Action was generously created externally by our OSS customer, Daniel at WLAN-Pi - please use at your own discretion.

To push packages to packagecloud using GitHub Actions you can:

1. Create an encrypted secret named PACKAGECLOUD_TOKEN in your GitHub repository and set your packagecloud API token as the value.
Just an example; get your real API token here.

gh secret set PACKAGECLOUD_TOKEN -b"f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0"

2. Click the 'Use latest version' button for the Upload to Packagecloud.io GitHub Action and add the snippet to your workflow .yml file.
Example snippet for v1:

      - name: Upload package PackageCloud.io
        uses: danielmundi/upload-packagecloud@v1

3. Under this step in your workflow .yml file, indicate your package name, username, repo to push the package into, OS/version of your choice (if applicable), and API token in the fields of this snippet:

        with:
          PACKAGE-NAME:
          PACKAGECLOUD-USERNAME:
          PACKAGECLOUD-REPO:
          PACKAGECLOUD-DISTRIB: 
          PACKAGECLOUD-TOKEN:

4. Push the file to your GitHub repo, and observe the workflow run under the 'Actions' tab from the GitHub UI.

  name: Push to packagecloud
  on: [push]
  jobs:
    Push-to-packagecloud:
      runs-on: ubuntu-latest
      steps:
        - name: Check out repository code
          uses: actions/checkout@v2
          
        - name: Upload package PackageCloud.io
          uses: danielmundi/upload-packagecloud@v1
          with:
            PACKAGE-NAME: packagecloud-test-1.1-1.x86_64.rpm
            PACKAGECLOUD-USERNAME: username
            PACKAGECLOUD-REPO: repository
            PACKAGECLOUD-DISTRIB: fedora/34
            PACKAGECLOUD-TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}

Questions? We're here to help! Email support@packagecloud.io

To deploy artifacts using the maven-packagecloud-wagon you must have at least Maven 3.x.x installed.

1. If you haven't already, add your API token to your local maven settings file located at $HOME/.m2/settings.xml. Optionally, you can encrypt these values for greater security, consult their Password Encryption documentation for instructions.

   Login to packagecloud to view your username and API token.

2. Add the dependency to the <build/> section of your project'spom.xml.

   <build>
     <extensions>
       <extension>
         <groupId>io.packagecloud.maven.wagon</groupId>
         <artifactId>maven-packagecloud-wagon</artifactId>
         <version>0.0.6</version>
       </extension>
     </extensions>
   </build>
   

3. Configure your project's <distributionManagement/> to point to your packagecloud.io repository, making sure that the <id> matches the entry we created above with your token. (We are setting our SNAPSHOT and release repositories to the same repository, feel free to use different ones according to your needs).

   <distributionManagement>
     <repository>
       <id>packagecloud-myrepo</id>
       <url>packagecloud+https://packagecloud.io/test-user/myrepo</url>
     </repository>
     <snapshotRepository>
       <id>packagecloud-myrepo</id>
       <url>packagecloud+https://packagecloud.io/test-user/myrepo</url>
     </snapshotRepository>
   </distributionManagement>
   

4. Deploy your project to packagecloud!

   mvn deploy
   

Questions? We're here to help! Email support@packagecloud.io

Deploying artifacts using the lein-maven-packagecloud-wagon:

1. If you haven't already, add your API token to your local leiningen profiles file located at $HOME/.lein/profiles.clj. Optionally, you can encrypt these values for greater security, consult their GPG documentation for instructions.

   Login to packagecloud to view your username and API token.

2. Add the dependency to the :plugins section of your project'sproject.clj.

   :plugins [[lein-maven-packagecloud-wagon "0.0.1"]]
   

3. Configure your project's :deploy-repositories to point to your packagecloud.io repository. (We are setting our SNAPSHOT and release repositories to the same repository, feel free to use different ones according to your needs).

   :deploy-repositories [["releases" {:url "packagecloud-https://packagecloud.io/test-user/myrepo"}]
                         ["snapshots" {:url "packagecloud-https://packagecloud.io/test-user/myrepo"}]]
   

4. Deploy your project to packagecloud.io!

   lein deploy
   

Questions? We're here to help! Email support@packagecloud.io

Deploying artifacts using the Gradle Maven Deployer

1. If you haven't already, add your API token to your local gradle file located at $HOME/.gradle/gradle.properties.

   Login to packagecloud to view your username and API token.

2. Require the maven plugin in your project'sbuild.gradle.

   apply plugin: 'maven'
   

3. Setup the deployerJars in the configurations section of your project'sbuild.gradle.

   configurations {
       deployerJars
   }
   

4. Add the dependency to the dependencies section of your project'sbuild.gradle.

   dependencies {
       deployerJars "io.packagecloud.maven.wagon:maven-packagecloud-wagon:0.0.6"
   }
   

5. Configure your project's uploadArchives to point to your packagecloud.io maven repository.

   uploadArchives {
       repositories.mavenDeployer {
           configuration = configurations.deployerJars
           repository(url: "packagecloud+https://packagecloud.io/test-user/myrepo") {
               authentication(password: mavenPassword)
           }
       }
   }
   

6. Deploy your project to packagecloud.io!

   gradle uploadArchives
   

NOTES:

Android Studio/Gradle 4.x.x Users: If you are getting errors like "Unable to detect maven coordinates (groupId, artifactId, version)..." while uploading, you'll need to define the coordinates explicitly inside of uploadArchives, like so:

uploadArchives {
    repositories.mavenDeployer {
        pom.groupId = 'com.my_group_id'
        pom.artifactId = 'myexamplelibrary'
        pom.version = '1.0.1-SNAPSHOT'
        configuration = configurations.deployerJars
        repository(url: "packagecloud+https://packagecloud.io/test-user/myrepo") {
            authentication(password: mavenPassword)
        }
    }
}

For SNAPSHOT support, just add -SNAPSHOT to your version.

You might see some "Could not find metadata: maven-metadata.xml" messages fly by; these are not errors and can safely be ignored.

Questions? We're here to help! Email support@packagecloud.io

Deploying artifacts using the Gradle Maven Publish Plugin Deployer

1. Require the java and maven-publish plugins in your project's build.gradle.

   ['java', 'maven-publish'].each {
       apply plugin : it
   }
   

2. Add your custom group and project version. For SNAPSHOT support, just add -SNAPSHOT to your version.

   group = 'io.packagecloud.sample'
   version = '1.0.0'
   repositories {
       gradlePluginPortal()
   }
   

3. Once the appropriate plugins have been applied, you can configure the publications and repositories. Don't forget to add your username and repository name into the url, paste your API token into username and leave password blank.

   publishing {
       publications {
           myPublication(MavenPublication) {
               // telling gradle to publish project's jar archive (2)
               from components.java
               // telling gradle to publish README file (2)
               // artifact ('README.txt') {
               //     classifier = 'README'
               //     extension  = 'txt'
               // }
           }
       }
       // telling gradle to publish artifact to repo (3)
       repositories {
           maven {
              url "https://packagecloud.io/test_user/test_repository/java/maven2/"
              credentials {
                  username = "<YOUR_API_TOKEN>"
                  password = ""
              }
           }
       }
   }
   

4. Deploy your project to packagecloud.io!

   ./gradlew publish
   

Questions? We're here to help! Email support@packagecloud.io

To deploy artifacts using SBT, we'll be wrapping our maven-packagecloud-wagon with the aether-deploy SBT plugin.
NOTE: You must have at least SBT 0.13.8 installed.

1. If you haven't already, add your API token to your local ivy credentials file located at $HOME/.ivy2/.credentials.

   Login to packagecloud to view your username and API token.

2. Add the plugin dependency to a project/plugins.sbt file in your project. (Create file if neccesary).

   addSbtPlugin("no.arktekk.sbt" % "aether-deploy" % "0.20.0")
   libraryDependencies += "io.packagecloud.maven.wagon" % "maven-packagecloud-wagon" % "0.0.6"
   

   NOTE: Only works with Java 8 or above, for Java 7 you'll have to use "aether-deploy" % "0.17", which does not support SBT 1.1.x

3. Paste the following block of code towards the bottom of your build.sbt file. This configures the credentials path and sets the publishing destination for our artifacts.

   import aether.AetherKeys._
   
   credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
   
   aetherWagons := Seq(aether.WagonWrapper("packagecloud+https", "io.packagecloud.maven.wagon.PackagecloudWagon"))
   
   publishTo := {
     Some("packagecloud+https" at "packagecloud+https://packagecloud.io/test-user/myrepo")
   }
   

4. Deploy your project to packagecloud!

   sbt aether-deploy
   

Questions? We're here to help! Email support@packagecloud.io

You can find additional documentation about the packagecloud command line here and by running package_cloud help.

Installation

Once you get the package_cloud CLI tool installed, you can create repositories and push packages quickly and easily.

1. If you don't already have Ruby, you need to install it. You can find install instructions for most platforms here.
2. Install the package_cloud Ruby gem by running:

   > sudo gem install package_cloud 

The package_cloud CLI has built-in help that can be accessed by running:

> package_cloud help

Creation of .packagecloud file

$HOME/.packagecloud allows package_cloud CLI to authenticate your session and check your permissions. There are two ways to set this file up

Manual creation

1. Go to https://packagecloud.io/api_token and copy the auto-generated settings.

2. Save the content into a new file in your $HOME directory.

   Just an example; get your real settings here.

   echo '{"url":"https://packagecloud.io/", "token": "f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0"}'' >> ~/.packagecloud

Direct download

You can also download the configuration file directly from the site if you click the download button in https://packagecloud.io/api_token, don't forget to rename this file to .packagecloud and move it from your downloads to your $HOME directory.

Creating repos

There are two ways to create a new repo:

1. Using the web UI: https://packagecloud.io/repositories/new
2. Using the package_cloud CLI:

   > package_cloud repository create myrepo

If your plan allows you to create private repos, you can create one by:

1. Selecting Private in the web UI on the repo create page: https://packagecloud.io/repositories/new
2. Passing a flag to the package_cloud CLI:

   > package_cloud repository create myrepo --private

Pushing packages

You can push a package using the package_cloud CLI:

> package_cloud push username/myrepo /path/to/packagefile

Some package types can be associated to specific versions of particular Linux distributions. If the package type you are pushing needs to be associated with a particular version you will be prompted by the package_cloud CLI.

You can also specify the version in the command line, for example, to push a package to Ubuntu 12.04 Precise, you would use:

> package_cloud push username/reponame/ubuntu/precise /path/to/packagefile

Supplying/Overriding Maven Coordinates

Sometimes (like for "fatjars"), we are unable to automatically detect the maven coordinates for a particular package, so you'll have to supply it on the command line like so:

> package_cloud push username/reponame/java /path/to/package.jar --coordinates=com.mygroup:packagename:1.0.2

Uploading multiple packages and skipping errors

You may upload an entire directory of packages using a glob:

> package_cloud push username/myrepo /path/*.rpm

If you attempt to re-run this command after additional packages to the directory, it will fail once it tries to upload a package that already exists. You can use the --skip-errors flag to force the CLI to skip packages that that already exist and proceed to the next package:

> package_cloud push username/myrepo /path/*.rpm --skip-errors

Yanking packages (Delete a package)

You can remove a package from a repo by using the package_cloud CLI:

> package_cloud yank username/myrepo/[distro/version] packagename.ext

The packagename.ext must be the full filename of the package, including the extension.

Some packages (RPMs, Debs, Python packages, Node.js, and JARs) are pushed to a specific distribution and version. When those packages are removed, the distribution and version that should be removed must be specified.

Other package types (RubyGems) can be removed without specifying a distribution or version, as these do not apply to RubyGem packages.

Examples

Remove a package called jake-1.0-2.src.rpm from a CentOS 6 repo named myrepo:

> package_cloud yank username/myrepo/el/6 jake-1.0-2.src.rpm

Remove a package called redis_2.8.1-1_amd64.deb from an Ubuntu Precise (12.04) repo named myrepo:

> package_cloud yank username/myrepo/ubuntu/precise redis_2.8.1-1_amd64.deb

Remove a package called packagecloud_test-0.0.1.whl from a Python repo named myrepo:

> package_cloud yank username/myrepo/python packagecloud_test-0.0.1.whl

Remove a package called passenger-4.0.43.gem from a RubyGem repo named myrepo:

> package_cloud yank username/myrepo passenger-4.0.43.gem

Remove a package called jake-2.3.jar with the group com.groupidfrom a Maven repo named myrepo:

> package_cloud yank username/myrepo/java com.groupid/jake-2.3.jar

Remove a package called packagecloud-1.0.1.tgz from an NPM registry named myrepo:

> package_cloud yank username/myrepo/node packagecloud-1.0.1.tgz

Remove a scoped package called packagecloud-1.0.1.tgz from an NPM registry named myrepo:

> package_cloud yank username/myrepo/node @computology/packagecloud-1.0.1.tgz

Promoting packages (Move a package)

You can move a package to a different epo by using the package_cloud CLI:

> package_cloud promote username/myrepo/[distro/version] packagename.ext username/destination_repo

The packagename.ext must be the full filename of the package, including the extension.

Some packages (RPMs, Debs, Python packages, and JARs) are pushed to a specific distribution and version. When those packages are moved, the distribution/version must be specified and they will be moved to the same distribution/version in the destination repository.

Other package types (RubyGems and NPM packages) can be moved without specifying a distribution or version, as these do not apply to RubyGem or NPM packages.

Both repository owners and collaborators may move packages between repositories.

This operation is a move, not a copy, thus the source repository will no longer have a copy of the package when the operation has completed. Note that the download statistics for the moved package will be cleared.

Examples

Move a packaged called jake-1.0-2.src.rpm from a CentOS 6 repo name myrepo to repo2:

> package_cloud promote username/myrepo/el/6 jake-1.0-2.src.rpm username/repo2

Move a package called redis_2.8.1-1_amd64.deb from an Ubuntu Precise (12.04) repo named myrepo to repo2:

> package_cloud promote username/myrepo/ubuntu/precise redis_2.8.1-1_amd64.deb username/repo2

Move a package called packagecloud_test-0.0.1.whl from a Python repo named myrepo to repo2:

> package_cloud promote username/myrepo/python packagecloud_test-0.0.1.whl username/repo2

Move a package called passenger-4.0.43.gem from a RubyGem repo named myrepo:

> package_cloud promote username/myrepo passenger-4.0.43.gem username/repo2

Move a package called packagecloud-1.0.1.tgz from an NPM registry named myrepo:

> package_cloud promote username/myrepo/node packagecloud-1.0.1.tgz username/repo2

Move a scoped package called @computology/packagecloud-1.0.1.tgz from an NPM registry named myrepo:

> package_cloud promote username/myrepo/node @computology/packagecloud-1.0.1.tgz username/repo2

Installing a repo

There are two methods you can use to install a repo:

1. You can visit the repo page and click the installation tab on the left side to get Bash scripts, a Chef cookbook, a Puppet module, and manual installation instructions.
2. You can also use the package_cloud command line client. You will need to specify the type of repo you want installed on your system (deb, rpm, or gem). For example, to install a deb repo you would run:

   > package_cloud repository install username/reponame deb

OS versions for pushing and yanking packages

When pushing or yanking a package you can specify an OS and a version. If you do not specify one, the command line client will walk you through choosing the correct OS and version.

The following table shows the OS and version strings that are valid for pushing and yanking:

RubyGems

No OS or version is required. RubyGem packages may be pushed like this:

package_cloud push user/repo example-1.0.1.gem

Node.js

NPM packages may be pushed like this:

package_cloud push user/repo/node example-1.0.1.tgz

Python

Python wheels, eggs, and source distributons can be pushed by specifying python. Python packages may be pushed like this:

package_cloud push user/repo/python example-1.0.1.whl

Amazon Linux

Packages for Amazon Linux must use the string for Enterprise Linux 6 (el/6) when pushing, or yanking, from a packagecloud repository.

package_cloud push user/repo/el/6 amz-linux-package-1.0-2.el6.x86_64.rpm

package_cloud yank user/repo/el/6 amz-linux-package-1.0-2.el6.x86_64.rpm

package_cloud push user/repo/{{value[0]["os_dist"]}} {{ packageFileName(value[0]["package_type"]) }}

package_cloud yank user/repo/{{value[0]["os_dist"]}} {{ packageFileName(value[0]["package_type"]) }}

Requirements

In your identity management service

To implement SSO login with packagecloud, customers must create a new application in their identity management service (Okta, Google Admin etc.), with the following details:

1. ACS URL:

   https://dev-px9pgfj4.us.auth0.com/login/callback?connection={email_domain}

   Example: If the email domain is packagecloud.io, the URL should be: https://dev-px9pgfj4.us.auth0.com/login/callback?connection=packagecloudio (without '.')

2. Entity ID:

   urn:auth0:dev-px9pgfj4:{email_domain_no_special_chars}

   Example: urn:auth0:dev-px9pgfj4:packagecloudio (without '.')

3. Add mapping attribute:

   Please map your email attribute to an app attribute called email.

   In some cases the default email field has a prefix, so we are normalizing it. For instance: primary_email → email

Update packagecloud admins

To enable SSO-login to packagecloud, please send these details to our team member you are in contact with, or support@packagecloud.io:

1. Your application's SSO URL,
2. Your application's x509 certificate file,
3. The SSO login-enabled person's/people's email (used to register with packagecloud),
4. Whether the user is an SSO admin (if applicable)

We appreciate your patience whilst our admins confirm the details and update the user's account.

Okta

Okta Configuration

The following steps should be completed by an Admin user of your Okta account in the Admin UI.

Settings for creating a new SAML 2.0 Application

• Single sign on URL

  https://dev-px9pgfj4.us.auth0.com/login/callback?connection={email_domain}

  Example: If the email domain is packagecloud.io, the URL should be: https://dev-px9pgfj4.us.auth0.com/login/callback?connection=packagecloudio (without '.')

• Entity ID

  urn:auth0:dev-px9pgfj4:{email_domain_no_special_chars}

  Example: urn:auth0:dev-px9pgfj4:packagecloudio (without '.')

• Name ID format: Unspecified

• Application username: Email

• Attribute Statements:

  • Name: email
  • Name format: Unspecified
  • Value: user.email

Add/assign new users to the account

1. Create a new Person

   In Directory (side nav) → People: Click the 'Add Person' button, enter the details, check the 'Send user activation email now' box and Save.

   NOTE: The inputted email should match the email used to register their packagecloud account.

   They will receive an email with prompts to create an account in Okta, and their Status will remain as 'Pending user action' until complete.

2. Assign the user to the application

   When the new user's Status is confirmed 'Active', go to your Application's Assignments (Applications (side nav) → Applications → <your_app_name> → Assignments), click the 'Assign' button, and 'Assign to People' from the dropdown menu.

   Select the user, click 'Assign' next to their name.

   Confirm the correct User Name, which should be the email used to register their packagecloud account, click 'Save and Go Back'.

Update packagecloud admins

To enable SSO-login to packagecloud, please send these details to our team member you are in contact with, or support@packagecloud.io:

1. Your application's Identity Provider Single Sign-On URL,
2. Your application's x509 certificate file,
3. The SSO login-enabled person's/people's email (used to register with packagecloud),
4. Whether the user is an SSO admin (if applicable)

The first two can be confirmed in the Okta Admin UI under Applications (side nav) → Applications → <your_app_name> → Sign On → View Setup Instructions

We appreciate your patience whilst our admins confirm the details and update the user's account.

Please refer to our Billing & Pricing FAQ for detailed info.

Package not found

• APT

There's two main reasons why this can happen:

1. The client system is using a CPU achitecture for which there is no matching package. For example, you've uploaded a package with architecture amd64 to your repository, but your client system is running i386. When this happens, APT will report that no package is found.
2. The local cached metadata on your system is stale. You can update it by running:

   sudo apt-get clean && sudo apt-get update

• YUM

For RPM based systems, this typically happens because the local cached metadata on your system is stale.

You can refresh it by running

sudo yum clean all && sudo yum update

You can also set

metadata_expires=[time in seconds]

in your repository config (found in /etc/yum/yum.repos.d/) to smaller value to allow for YUM to automatically refresh the metadata. See the man page (man 5 yum.conf) for more information.

GDPR

The General Data Protection Regulation (GDPR) is a new European data protection regulation adopted by the EU Commission, which aims to strengthen the security of personal data.

Below is some information about the steps Computology is taking to meet the GDPR requirements, and how we are working with our EU customers to help them meet their compliance obligations.

Is Computology a data controller or data processor?

In general, Computology is a data controller for personal data derived on its website, and personal information such as contact information collected directly from our EU customers.

Computology is a data processor when its EU customers collect the personal information of their employees or customers and upload that personal data to their Computology account. In this case, the EU customer is the data controller for that personal information.

What are customers’ responsibilities as a data controller?

EU customers who have collected the personal information of their employees or customers need to ensure that they have legal consent of those parties under the GDPR, and that there is an adequate level of protection for that data exported out of the EU.

Does Computology have a Customer DPA?

Yes, it does. Contact us at support@packagecloud.io and it will be provided to you. Also Computology enters into standard contractual clauses with EU customers upon request.