ACG Website: move from HostMonster to GitHub, layout and settings

[RedshiftDrift]

I am trying to find the quickest way to change the appearance of the website so that there will be an index of the site in a separate column that will show the folder structure.

I started reading the manual Quickstart for GitHub Pages where they have the settings for the webpage in the file _config.yml. However in our repo the file is called _config.yaml.

Is there a reason for the different name?

[RedshiftDrift]

Never mind, I found the answer!

[ftessier]

I think both extensions work (?) I don't even know which one I prefer 😄, .yaml is certainly unequivocal.

[ftessier]

I think the easiest is to find a remote Jekyll theme and apply it, for example:
https://jekyllthemes.io/free

[ftessier]

This one looks like an interesting starting point: https://jekyllthemes.io/theme/just-the-docs

[RedshiftDrift]

It works!

Thanks! Your course has top priority, don't worry, ACG can wait!

The universe is older than 35 billiion years, one more day won't make any difference!

[RedshiftDrift]

Success! https://a-cosmology-group.github.io/acg/
Still a bit of tweaking to do (each page needs a YAML front matter) but otherwise it seems to work.

[ftessier]

Beautiful @marmetl!! I was wondering how to do that, I wanted something similar for the EGSnrc documentation. Thank you for showing the way. 🙏🏻

I can help convert some of the legacy .html to .md in a fairly automatic fashion (using pandoc). Let me know if some files ought to be converted in priority.

[RedshiftDrift]

Thanks @ftessier for the ~automatic conversion. There is no real priority, the cosmology.info url will be moved to the GitHub website when it is mostly ready.

[RedshiftDrift]

By the way @ftessier would it help you to have a .zip file of the entire website? I could post it on GoogleDrive...

[ftessier]

This is provided by github too! You can do that with a "Release". See the EGSnrc releases for an example.

When you create a "Release", the entire repository is attached as both a .zip and a .tar.gz, under the "assets" section. Anyone can download the compressed archive(s).

You can structure Releases as you wish. For a website, it stands to reason that you would one maintain a single "Release", which would always be kept up to date.

Let me know if you want to experiment with that right away, or set this aside for a rainy day!

[ftessier]

I still don't understand... Ah, you mean a zip of the original website, right? Yes, it is a good idea to keep this around, especially over the transition, if anyone wants to check on a bit that might not have been converted properly. I would attach the .zip file to a Discussion here though, so it is all associated with the repo.

[ftessier]

Or, if it is not too large (strip .pdf and other binaries?), we can commit it inside the repo itself, for the record. But since the files therein won't change, there is little value in versioning them, apart from access in the repo itself.

[RedshiftDrift]

No, I am still not getting my message through to you!

All the files for the website cosmology.info (the original, not the GitHub version) are on my computer. Copies of those files are on the cosmology.info server, so they are seen on the website cosmology.info. In the past weeks, you have been taking files from cosmology.info, converting them with pandoc and committing them to the a-cosmology-group repo.

I am offering to:

1. Generate a cosmology_info.zip file of the original website from my computer with everything in it (contains .html, .jpg, .pdf, .csv, .txt and .zip),
2. Post cosmology_info.zip on my Google drive and give you (only you) access to it for your private use.

This would allow you to:

1. Download the cosmology_info.zip file it to your computer (not to a Git repo, no versioning of that file is required),
2. Unzip cosmology_info.zip to have access to the folder structure of cosmology.info on your computer.

So far, nothing has been committed on GitHub yet 😄 ! Now, you could

1. Pick the cosmology_info files from your computer (instead of downloading them from the web) and convert the .html format to .md with pandoc,
2. Then upload the resulting .md files to the repo, as you have already done for some files.
3. Once you are done you can delete cosmology_info.zip.

I am offering this only if it makes easier for you to work this way instead of getting the files from the cosmology.info website.

The whole site is 103 MB big, most of the space is for files that won't change, so there's no need to commit a .zip file of the same website.

Only the .html files need to be converted to .md and committed to the a-cosmology-group/acg repo. I can upload the other formats (.jpg, .pdf, .csv, .txt and .zip) directly to the a-cosmology-group quickly since no conversion is necessary. (See a-cosmology-group/acg/newsletters/assets, it took 10 seconds to upload that!)

[ftessier]

Oh gosh @marmetl I am sorry I made you write this long explanation! The reason I seemed so clueless is because I already have all the files, and I assumed you knew that. So I automatically thought you meant "for other people", not just me (even though you said "would it help you"). Apologies. I used the following terminal command to download the entire website:

wget -r http://cosmology.info/

[RedshiftDrift]

No problem, I did not know that you could "download" the entire site 😮 with one command!

[RedshiftDrift]

I uploaded several 10s of MB of .pdf files and squash and merged the whole thing.
Every file that's not in .md format is placed in assets.
README.md files are for information about the folder, while index.md files are for the website's index.html.

I am not exactly following the folder structure of the original website. The folder models will take the webpages from individual members who want to create them. @ftessier you can merge Claudio README.md but rename it index.md if possible, and don't merge the pdf file for now (if that's possible). See my comment there.

The website https://a-cosmology-group.github.io/acg/ is posted correctly but some error messages are generated. I will look into that later.

[ftessier]

Alright, thanks for letting me off the hook. 🤓

If you want, don't hesitate to prompt me ahead of a merge, so that I can "clean up" the pull request branch (typically squashing the many little "irrelevant" commits together). Up to you, makes it easier to browse the history, but has no impact on content per se.

If and when I do, I will detail the operations, so that everyone can learn how to do this!

[RedshiftDrift]

I use "squashing" when I merge many edits to many files. Can you clean this (e.g. this one) even more?

[ftessier]

Sorry, I had not seen that you already squashed these pull request branches, I see now from the commit history on main that the PRs were indeed squashed, all good. I cannot reduce it below one commit.

However, let me know if you want to cleanup the branch without necessarily squashing everything together. For example, you might want to get rid of some noise due to trial and error, but still retain 3 separate commits for 3 logically separate updates. It is possible to combine and reorder commits at will, mostly using the git rebase -i and git cherry-pick terminal commands. I do this every day, so I got the hang of it.

Otherwise, squash away as you have been doing so far! 👍🏻

[RedshiftDrift]

Thanks, I'll remember that you can do all that!

At this point I am still going through tests and none of the earlier commits are of any use. It's not like developing code, these files are only webpages with typos, so you could squash the entire main acg branch and no useful information would be lost!

B.t.w. I did add some javascript code - I moved my cosmology calculator here 😄

[RedshiftDrift]

For a week now I have been editing files on by computer and pushing with the terminal (non text-files can't be renamed/moved using the web interface 😠 )

I use a limited set of commands:

$ git pull
$ git status

then I modify/edit several files, and then

$ git add .
$ git commit -m "Test Jekyll/just-the-docs/MathJax and edit website"
$ git push origin main

... and then I check the workflow run on the GitHub web interface and the updated website!

@ftessier I just realize that $ git push origin main doesn't squash, so you may want to clean up the many commits done in the last week!

My question is, what would be the git command to push and squash?
Is there something like $ git push --squash origin main I can use?

[oposay]

very informative

[ftessier]

@marmetl You are really getting the hang of this!

Indeed, working on the github interface (pull request, issues, etc.) is useful to interact with the community who submit ideas or bugs. But working with git on a local clone and pushing to the repo is much more convenient. There are software guis that are very nice to work locally, if the terminal is not your thing (VS Code with github integration, GitKraken, etc., there is a zoo of them).

For squashing in the terminal, the command you are looking for is git rebase --interactive [base], where [base] is the commit hash on which you want to "rewrite history".

Git rebase is a slightly complex commands, as it is a two-step process. You are first presented with a list of commits from the rebase [base] to the current commit, in your text editor, in chronological order. For example, main is now 9da3c86, and if I git rebase -i 8000bb8, my editor opens with the following content (instructions included!):

pick 3ba123f Add gravity models navigation
pick 82c0dcc Test line breaks
pick a1b1d45 Fix equation
pick 0fe9c37 Add documents to
pick eb811c9 Fix web pages (#73)
pick 5104a6f Pull again? (#74) # empty
pick 9da3c86 Edit Non Expanding Cosmologies webpages

# Rebase 8000bb8..9da3c86 onto 8000bb8 (7 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup [-C | -c] <commit> = like "squash" but keep only the previous
#                    commit's log message, unless -C is used, in which case
#                    keep only this commit's message; -c is same as -C but
#                    opens the editor
# x, exec <command> = run command (the rest of the line) using shell
# b, break = stop here (continue rebase later with 'git rebase --continue')
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
#         create a merge commit using the original merge commit's
#         message (or the oneline, if no original merge commit was
#         specified); use -c <commit> to reword the commit message
# u, update-ref <ref> = track a placeholder for the <ref> to be updated
#                       to this position in the new commits. The <ref> is
#                       updated at the end of the rebase
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#

(3ba123f is the first commit after 8000bb8).

This is where you tell git how you want to rewrite the commit history: by changing the first word on the line for each commit (initially pick for all). In terms of squashing everything, you can change pick to squash (or simply s) for all commits except the first one (since you need a commit to squash "onto".

Upon saving and quitting the editor, git will proceed with rewriting the history accordingly. When squashing, you will be presented with the opportunity to edit the squashed commit message, which by default is a concatenation of all the messages from the squashed commits.

There are many other options in rebase, like reordering commits (by reordering the lines), reword the commit message (useful for typos), edit commit to change the content of the commit (say, adding files or tweaking the commit content). Note however that it can lead to conflicts. If things get out of hand, you can always issue git rebase --abort to go back to the original branch (before the rebase was initiated). Squashing (without reordering) never leads to conflicts.

Finally, "rewriting history" is not recommended if you have already shared the commits publicly: collaborators might have started forks or modifications on the existing commits. Due to the "cryptographic blockchain" nature of the commit references (hashes of changes, metadata, and the parent commit hash), rebasing implies that all rebased commits will acquire a new commit reference. That is why it is a good practice to work on side-branches, and merge the resulting commit or branch into main, which itself is never rebased so that refs are permanent and always valid on main.

[RedshiftDrift]

Thanks @ftessier , it works and generates the commands automatically which are executed once I close the file! That's quite a powerful tool.

$ git rebase --interactive
hint: Waiting for your editor to close the file...
unix2dos: converting file C:/Users/Louis.NEURONE10/Sync/Websites/a-cosmology-group/.git/rebase-merge/git-rebase-todo to DOS format...
dos2unix: converting file C:/Users/Louis.NEURONE10/Sync/Websites/a-cosmology-group/.git/rebase-merge/git-rebase-todo to Unix format...
Successfully rebased and updated refs/heads/main.

However, in practice, because I am testing Jekyll/just-the-docs/MathJax on the acg(main)-branch, I have only one commit to squash on my local repo. So I only had one command line and no squashing to do. I understand why "rewriting history" would not be a good idea.

This is way more than I signed up for! I only wanted to transfer the website and get this to work automatically after a push. I won't be using this in the future, so unfortunately most of it will be forgotten in a few months... (I'll have my notes.)

[ftessier]

This is way more than I signed up for! I only wanted to transfer the website and get this to work automatically after a push. I won't be using this in the future, so unfortunately most of it will be forgotten in a few months... (I'll have my notes.)

No doubt more than you signed up for! 😄 But throught it all you have taught me how to build a proper website on github, so thank you! I can remain on the sideline to help with git if needed.

The strategy I found to not forget git over time is to use it for all my projects. In roughly a decade of doing that, I have not yet found a situation where that has not been beneficial. It is a little cumbersome at first, but the work-add-commit-work-add-commit-... dance quickly becomes second nature.