I don't add content to this site very often. I should. Part of the reason for that is I don't actually rememember how to get content to appear here.
When I first setup this system up, I had it so that any content I checked into github would automatically regenerate the site and upload here. At some point that system stopped working, and my knowledge of how it was all setup atrophied.
I am going to attempt to document my progress of getting it working again here. I'll hopefully be able to use this docunent as a reference in the future.
First to specify what I know:
Initial clicking around appears to suggest usage of the "Compute Engine API".
I clicked around a bunch and came up empty. Then I remembered that it might actually be based on Firebase instead of Google Cloud directly. Clicking through to the console seems to confirm this.
I am going to pause here for awhile. I think I need a personal computer with command line access to continue.
Whew, it's been awhile!
The next thing to try was downloading a copy of the site locally. I have a new (to me) OSX laptop. I ran "git" but it was not installed. OSX prompted me to install it automatically, and I did.
I had to setup an SSH key to actually work on the code. I could have cloned the repository with out it, but I won't be able to push. I started with GitHub's instructions for adding an ssh key to an account. This, of course, meant generating an ssh key first. Thankfully, github has an article for that as well.
ssh-keygen -t ed25519 -C "your_email@example.com"
Followed by
eval "$(ssh-agent -s)"
Then adding the following to ~/.ssh/config
Host github.com
AddKeysToAgent yes
UseKeychain yes # omit line if not using a passphrase
IdentityFile ~/.ssh/id_ed25519
Then
ssh-add --apple-use-keychain ~/.ssh/id_ed25519
And lastly
pbcopy < ~/.ssh/id_ed25519.pub
This last command copied the public key to my clipboard. I could then paste it into the new ssh key page and save it.
WHEW!
I cloned the repository locally, where I am editing it now.
The next step was getting python up and working. This blog is generated by Pelican. I am actually a little terrified that new versions of Pelican may actually not be backwards compatible, but one thing at a time!
Python does not come installed on OSX by default. Some quick googling suggested that Homebrew was the best way to install and manage Python these days. I copied and pasted the install command at the top of the page:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Always a little terrifying to curl random bash scripts, but I suppose it's no worse than downloading and running any other random, uninspected binary.
Next was "brew install python". This installed Python 3.13.2. It did not, however, add "python" to the path, only "python3". Documentation I found suggested adding
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
to my "~/.profile" file, but this didn't work. OSX runs zsh by default, so I needed to add it to "~/.zprofile".
Next to setup a virtual environment to run pelican in.
pip install --user pipenv
Homebrew intercepted me here and told me it was an error to do so. Instead, it suggested
python3 -m venv path/to/venv
I ran "python3 -m venv ." in my repository but this seems to have been a mistake. It edited my ".gitignore" file to ignore everything. I restored ".gitignore" and saw that it had had added several files and directories to the repository. Time to try something different.
I made a new directory, which will server as my vitual environent, and moved my repository in as a subdirectory. I then reran the venv command, and everything seems happier now. I ran
. bin/activate
to enter my vitual environment.
Pelican suggests installing it with
python -m pip install "pelican[markdown]"
pip then informed me that I should upgrade it, so I did.
Pelican says to run
pelican -r -l
So I changed to my repositories subdirectory, ran it, and was met with a bunch of errors.
[21:03:45] WARNING Feeds generated without SITEURL set properly may not be valid settings.py:679 WARNING No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check settings.py:684 https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information --- AutoReload Mode: Monitoring `content`, `theme` and `settings` for changes. --- Feeds generated without SITEURL set properly may not be valid No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information Process Process-1: Traceback (most recent call last): File "/usr/local/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap self.run() ~~~~~~~~^^ File "/usr/local/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/dave/devel/othm/lib/python3.13/site-packages/pelican/__init__.py", line 572, in autoreload settings_file = os.path.abspath(args.settings) File "", line 378, in abspath TypeError: expected str, bytes or os.PathLike object, not NoneType Serving site at: http://127.0.0.1:8000 - Tap CTRL-C to stop
The SITEURL and timezone matters I can fix. I am currently unsure about the stacktrace however. Visiting the site as it suggests also fails.
Back to it, the reason for the above errors is the version of Pelican this site was originally compiled with (2? 3?) ran on Python 2, while Pelican 4 only supports Python 3. I have a choice: install Python 2, and then install an old version of Pelican, or just upgrade this site.
Time to upgrade this site!
First, I created a new Pelican site alongside the existing one, just to see what's changed. After creating a new emptry directory, I run "pelican-quickstart" per the instructions, but the command doesn't work. I see the the command exists in my virtual environment's "bin" directory, but that bin is not in my path. I run "../bin/pelican-quickstart" instead.
I followed the on screen prompts with mostly default or obvious values. When I got to the timezone option, it defaulted to "Europe/Rome", which is not correct, so I started googling around for the appropriate string (something related to us eastern timezone). The first Wikipedia article on the Eastern Time Zone does not appear to contain a magic string that looks correctly formatted. I am pretty sure I am looking for something like "New York/US" or similar. Next I found Time.is which contains a bunch of "IANA time zone identifiers" which look correct. I selected "America/New_York". After a few more questions where I selected the defaults, my new site was setup.
pelican content
pelican --listen
With those two commands, I had a working, if bare, Pelican site. I stopped the pelican process.
I copied over the "content" directory of this existing site into the content directory of this temporary site and ran "pelican -l -r". I was greeted by many errors, including an apparent infinite loop:
Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' There are 2 items "with slug "2016-in-review"" with lang en: /Users/dave/devel/othm/tmp/content/articles/2016/2016_in_review.html /Users/dave/devel/othm/tmp/content/articles/2022/butterfly.html There are 2 original (not translated) items with slug "2016-in-review": /Users/dave/devel/othm/tmp/content/articles/2016/2016_in_review.html /Users/dave/devel/othm/tmp/content/articles/2022/butterfly.html {filename} used for linking to static content /images/2016/parachute-skydiving-parachuting-jumping-38447.jpeg in articles/2016/double_blind_parachute_trial.html. Use {static} instead Caught exception: "File /Users/dave/devel/othm/tmp/output/2016-in-review.html is to be overwritten". Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date' ...
First, I fixed the metadata in butterfly.html, changing the title, keywords, and date. Then I added the time to the the_magical_wordomatic.html, hoping that would resolve the loop. I reran "pelican content" and it appears to have worked, outside of a few deprecation warnings. I then ran "pelican -l -r" again and it appeared to be working site.
Then I looked over some of the earlier output and realized that I still have an error in the_magical_wordomatic.html page:
ERROR Skipping /Users/dave/devel/othm/tmp/content/word-o-matic/index.html: could not find information about 'date'
ThenI looked closer and realized that all my articles are actually in "content/articles" and that it is trying to parse what it supposed to be a static page.
The documentation says that setting "STATIC_PATHS" in "pelicanconf.py" should be enough to fix it. I copied over the values from the old version of the site, but the error persisted. After a bit of playing around and making no progress, I decided to move on. Maybe this was happening because my output structure was flat, instead of nested directories as it is currently?
I copied and pasted over the ARTICLE_SAVE_AS and ARTICLE_URL values from the previous site. This seems to have organize things, but still no luck on the word-o-matic.
Finally, I changed "ARTICLE_PATHS" from its default value to "['articles']" and this seems to have resolved the issue.
I copied over the "theme" directory next, and blindly set the "THEME" property to "theme/othm" as I had in the prioer version and was met with a new error: "Encountered unknown tag 'assets'.". Some plugin I am missing perhaps? I copied over the "plugins" and "pelicanplugins" directories, added them to the pelicanconf.py file and...
[11:53:21] ERROR Cannot load plugin `assets` Cannot import plugin `assets` ERROR Cannot load plugin `gzip_cache` Cannot import plugin `gzip_cache` ERROR Cannot load plugin `htmlminify` No module named 'htmlmin' ERROR Cannot load plugin `optimize_images` Cannot import plugin `optimize_images` ERROR Cannot load plugin `summary` Cannot import plugin `summary` ERROR Cannot load plugin `tag_cloud` Cannot import plugin `tag_cloud` --- AutoReload Mode: Monitoring `content`, `theme` and `settings` for changes. --- Cannot load plugin `assets` Cannot import plugin `assets` Cannot load plugin `gzip_cache` Cannot import plugin `gzip_cache` Cannot load plugin `htmlminify` No module named 'htmlmin' Cannot load plugin `optimize_images` Cannot import plugin `optimize_images` Cannot load plugin `summary` Cannot import plugin `summary` Cannot load plugin `tag_cloud` Cannot import plugin `tag_cloud` Serving site at: http://127.0.0.1:8000 - Tap CTRL-C to stop Caught exception: "Encountered unknown tag 'assets'.".
I cloned the old pelican-plugins which is evidently deprecated in favor of the newer pelican-plugins github org. I'll sort that all out later.
Still, a few plugins were broken: htmlmin, summary, and assets. I disabled the htmlmin plugin. Summary asked for the "bs4" package to be installed, which I did. Assets needed the "webassets" package.
Now running "pelican content" almost works, save for a new error about "cssmin" not being installed. After installing that, things maybe worked, except now I get an error agout "jpegtran" and "optipng" not being installed, although the content says it generated succesfully. Loading the site in a browser now sort of works, though css appears incorrect. Viewing the network traffic does not show any 404 errors. Maybe I inlined the css?
Looking at the source for the output site, I indeed saw an empty style tag at the top. The theme is expecting some sort of "ASSET_CONTENTS" variable to be populated, but I can't see this documented anywhere in the asset's plugin documentation.
Instead of inlining the content, I uncommented a link tag I had at the top and that seems have to restored things. I think I am getting close!
Clicking on tag on the side brings me to a different (though functional) url than is live on the site today. I scanned through the difference in the settings file and found that TAG_URL and PAGE_URL were set on the prior version of the site, so I copied those over. Things seems very similar now!
There are only two obvious differences: First, the dates are displaying differently on the articles, but I am ok with that. I will probalby change it to something different in the future. Second, the new site has the front page paginated. I have wanted to do this for awhile, but not as it is currently implemented. Setting "DEFAULT_PAGINATION" to false fixes this up.
Ok, so I had a newly generated site. I copied it over the old content, ran git diff, saw nominal differences, and commited.
But now I still need to push to the server.
I found documentation "Get started with Firebase Hosting". Step 1 is to install the Firebase CLI tools. Then I logged in an tested the tooling, ala "firebase login" command.
Inside my git repository, where pelican is actively running, I ran "firebase init hosting". I chose an existing project, pointed at pelican's output directory, and left everything else at the defaults.
At this point, if I run "firebase deploy --only hosting", I might have a working site? However, I don's want to deploy over the existing site until I have verified that this will all work. There should be a way to deploy to a staging/testing location.
Reading the docs, it suggested running "firebase hosting:channel:deploy preview". This returned a url with the website hosted. Hooray! However, it also fails to render any css. Good thing I didn't push to prod. Opening up the dev console, it appears to be as simple as the fact that the css was loaded over http instead of https.
I popped open "pelicanconf.py" and changed the SITE_URL to https. Then I noticed the the local development copy of my site was loading the css off of the web instead of a local copy of the css. This is wrong.
Uncommenting "RELATIVE_URLS" in pelicanconf.py fixed this up. I will also need to be careful to publish with the publishconfi.py file when actually pushing to firebase.
One more call to "pelican content -s publishconf.py" followed by another call to "firebase hosting:channel:deploy preview" and my site looks ok! However...
When I click over to the word-o-matic, the page loads and is functional, but if I click any of the "presets", I get a 404. This could just be a quirk of the preview domain.
But I did some more digging and discovered url rewrites. I don't know why I do not have this firebase.json file already checked into my git repository, but I am going to now! I added the following to my firebase.json:
"rewrites":[ {
"source": "/word-o-matic/*/",
"destination": "/word-o-matic/index.html"
} ]
Partial success. The page now loads, but the json containing the preset content returns a 404. Terrifyingly, I can' find those json files anywhere in my repo. Where are they?
I downloaded all the json files and added them to the repostiory. Another push to firebase and... partial success? For some reason, two of the json files are not working. "jabberwork.json" is returning the html for the word-o-matic page, and "prescription-drugs.json" is returning a 404.
Even stranger, I fired up firebase's local emulator and everything works fine locally. I deployed a second time to firebase without changing anything and... success!
The last, and most suspicious thing is that the current site says it contains almost 20 thousand files, while the site I just previewed contains only 370 files. I suspect this is because I used to allow folks to upload and save custom content in the word-o-matic, and I maintained that old content even after I disabled that functionality. I have no straightforward way of recovering that content at the moment, but it may simply be time to say goodbye to it. It is possible that that content continues to reside on the hard disk of another computer that is not currently hooked up, but that is a problem for another day.