admin
21 Oct 2022
My workflow for Drupal site upgrades and deployment.
 
In the old tarball and GUI era of managing Drupal, I simply made a backup copy of both the codebase and database and then did the Drupal upgrades (core, modules, or theme) on production, and hoped for the best. It is a testament to Drupal that this almost always worked without a problem. Drupal has evolved into a CMS or platform with enough complexity (multiple external packages, libraries, and dependencies) that this approach is consistently disastrous. (Someone said: If you are not using a dependency manager with Drupal, then YOU are the dependency manager - and you are not reliable.)
 
After some struggles, false starts, and missteps, I have come up with a system that works for me. I share it here, partly because it may help others, and partly as a reference for myself.
 
My context:
  • I add content to my production sites through the GUI interface, but I NEVER do updates or upgrades or add/remove modules on production.
  • I am not part of a team so I do not need GIT as a source of truth for multiple other team members. It would not be hard to add GIT to my process.
  • I currently have 7 production sites for non-profits, community organizations, and my own blog (right here). One site does commerce. Several of the sites are pretty static, but four involve posting a great deal of new content, and two involve user contributions and comments.
  • ALL my work is pro-bono.
  • I use shared hosting. (I use DreamHost. They are not Drupal-centric but I have been happy with their reliability, affordability, and support.)
  • My tools are: M1 MacBook Pro, Terminal and command line, DDEV, Composer, Drush, Transmit.
 
Overview of my process:
  • I create a local development copy named <my_site2> of my production site <my_site>.
  • I do and test all updates, upgrades, and changes/additions/subtractions to modules and themes locally locally on <my_site2>. On occasion, I upload the updated site to a staging domain before pushing it to the live production site, usually because I want feedback.
  • I push the updated/tested local codebase and database from <my_site2> to production in parallel to the live site  <my_site> and then use the control panel on the host to point to the updated site. This allows me to quickly and easily fall back to the pre-update version if necessary.
  • For each iteration, I create a new local development site that is numbered sequentially: <my_site3>, <my_site4>, etc.
  • The basic concept of my process is straightforward, but there are many steps and attention to detail is very important.
 
(NOTE: anything in <brackets> is to be replaced with appropriate site-specific information. Terminal commands are in italics. Project-root is the top-level directory for the project and contains a web-root with the site code-base.)
 
PHASE 1: Make a local copy of the production site:
  • On the production server, use Backup and Migrate (or phpMyAdmin) to create a compressed copy (database_name.sql.gz) of the production database and download it.
  • Log into the production server with a file transfer tool. (I use Transmit and shared hosting on DreamHost.)
    • Make a compressed archive of the production codebase using SSH or Terminal commands: tar -czvf <my_site2>.tar.gz  <my_site> .
    • Download the archived <my_site2.tar.gz>  and extract it tar -xvzf. <my_site2.tar.gz>.
    • NOTE: it will extract named as the original production <my_site>, not as <my_site2>.  I rename this directory <my_site2> BEFORE moving it into /Sites to avoid overwriting <my_site> in /Sites. In <my_site2/.ddev/config.yaml> change name:my_site to name:my_site2.
  • Use local Terminal, navigate to the downloaded and extracted directory (cd Sites/<directory_name>) and use DDEV to start a container for the local copy (ddev start)
    • (Note: if you haven't edited 'name' in .ddev/config.yaml you will not be able to start the project with DDEV.)
  • Import the downloaded and compressed database: ddev import-db —src=/path-to-downloaded-file. Or just use ddev import-db and it will prompt you to add the path (one can drag and drop it in Terminal.)
  • Launch the local copy: ddev launch (this opens the local copy in the default browser)
  • Verify that the new local development site is working before mucking with it. (Check reports.)
  • Run ddev snapshot.
 
 
PHASE 2: Update the local development copy
  • I use Terminal, DDEV, Composer, and Drush.
  • Navigate with Terminal to the project-root of your local development  /Sites/<my_site2>
  • Verify ability to update core with a recommended version: ddev composer show drupal/core-recommended. (The first line of the output should be drupal/core-recommended.)
  • Test the core update process: ddev composer update "drupal/core-*" —with-all-dependencies —dry-run
  • If dry-run shows no errors or incompatibilities (rarely a problem), do the core update: ddev composer update "drupal/core-*"—with-all-dependencies
  • Wait as it downloads and applies the new code and any changes in dependencies (libraries or other packages that are necessary). (Before DDEV, one had to do this manually, one dependency at a time.)
  • Run ddev drush updatedb to update the backend database (Or ddev drush updb )
  • Run ddev drush cache:rebuild to flush cached pages. (Or ddev drush cr )
  • Refresh the local browser tab that is open to the local development copy of <my_site2> and verify that core has been updated using Reports.
  • Run ddev composer outdated "drupal/*" to get a list of other pending updates (one can also see this through the website interface.)
  • IMPORTANT: check the release notes of the updates to be sure one has the correct version number and the correct way of specifying it. Sometimes it is best to defer an update.
  • Update all the modules using DDEV and Composer by running ddev composer require "vendor/module1:version"  "vendor/module2:version" "vendor/module3:version" and so on.  This will update the modules and their dependencies.
  • Run ddev drush updatedb (or ddev drush updb) to update the backend database. It will list the needed changes in the database and ask if it should go ahead. Type Y  for yes.
  • Run ddev drush cache:rebuild (or ddev drush cr) to flush cached pages.
  • Look at the section on settings.php under Glitches and Gotchas below. It is easier and better practice to fix this here than on production after uploading and finding an error.
  • Use the browser tab to verify that the site works. Check reports. (If there have been no errors thrown during the process, it pretty much always works.)
  • Create a database snapshot: ddev snapshot
  • (Optional: this would be the time to add or remove modules or themes. If you do, remember to repeat creating a database snapshot: ddev snapshot .)
 
 
PHASE 3: Push the updated copy to production:
  • The local development site is now fully patched and updated. It has all the current content from the production site because the content is stored in the database that was downloaded and imported. It is time to replace the current production site with the updated development site - without losing the ability to easily fall back to the old production site.
  • Create a compressed copy of the updated local development database: ddev export-db --file=/path/<my_site2>.sql.gz (Make sure you know where this is stored on your local computer. If you don't specify a path, the location defaults to .ddev in your project root directory.)
  • Use Terminal on the local computer and navigate to /Sites (cd Sites)
  • Create a compressed archive of the updated local development site: tar -czvf  <my_site2>.tar.gz and use the file transfer program to upload this to the production server at the same level as the currently active html or web directory. (Note: the actual web site codebase lives in a /web directory inside the project folder: <my_site2/web> ). Lots of support and other files are in the main ‘project’ directory (<my_site2> but above the level of the web site and therefore not visible to browsers and secure.)
  • Extract the archive of the codebase: tar -xvzf <my_site2>.tar.gz.  You now have a copy of the updated site's project including the codebase on the production server name <my_site2>.
  • On the production server, use phpMyAdmin to create a new empty MySQL database and import the exported and compressed copy of the local database into this empty database.
  • On the production site, using the file transfer program, navigate into the project-root directory <my_site2> and then to /web/sites/default/settings.php.  In that file, change the information about the name and credentials of the database to the new database. You will have to change the permissions on that file to edit it, and then change them back to read only (444) when done editing.
  • In the control panel of the host, edit the configuration so the website is accessible (changing a configuration option from <my_site>/web to <my_site2>/web) (This process will look and function somewhat differently on different hosts.)
  • Test. (Check Reports)
  • Assuming everything works, make a copy of the codebase and database for the revised site and store them somewhere safe and well labelled, as you will likely accumulate a fair number over time.
 
Glitches and gotchas:
  • Watch out for curly quotes in terminal commands
  • Don't remove the current production codebase or database from the host until the updated version is up and running. It serves as a fall back.
  • Make sure settings.php is 444 before compressing and uploading to production or immediately after you upload and extract the codebase.
  • Do a ddev snapshot before creating the tarball to upload
  • BEFORE uploading your development site <my_site2>, be sure to check settings.php for:
    • Hash: make sure hash is not an empty string in settings.php
    • /tmp directory
      • Create tmp directory in Drupal project above codebase (at same level as /web directory
      • Set path in settings.php: $settings['file_temp_path'] = $app_root . '/../tmp';
    • private directory
      • Create private directory in Drupal project above codebase (at same level as /web directory
      • Set path in settings.php: $settings['file_private_path'] = $app_root . '/../private';
    • sync directory ($settings["config_sync_directory"] = '../config';)
      • Create config/sync nested directory in Drupal project above codebase (at same level as /web directory
      • Set path in settings.php: $settings['config_sync_directory'] = $app_root . '/../config/sync';
 
 
 
Links to more on this topic:
Composer
DDEV
Drupal
Drush
Updating
Upgrading