A github actions example to extract issues of the repo itself

[0ut0fcontrol]

Thank you for your great work.

I add my github actions example here, in case someone needs it.

it backups all issues every day.

Because it only backup the issues belong to the repository that contains this workflow. It does not limit by the GitHub API rate.

# gh2md log:
...
Writing to file: issues.md
Github API rate limit: RateLimit(core=Rate(reset=2020-09-17 02:50:05, remaining=977, limit=1000))
Done.

# .github/workflows/issues2md.yml
name: Issues2Markdown
on:
  push: # comment it to reduce update.
  schedule:
    # every day
    - cron: "0 0 * * *"
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
    - name: Backup github issues to a markdown file.
      run: |
        pip3 install --user --upgrade setuptools
        pip3 install --user gh2md
        $HOME/.local/bin/gh2md $GITHUB_REPOSITORY issues.md --token ${{ secrets.GITHUB_TOKEN }}
        git add issues.md
    - name: Commit files
      run: |
        git config --local user.email "[email protected]"
        git config --local user.name "GitHub Action"
        git commit -m "Backup all issues into issues.md" -a
    - name: Extract branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: extract_branch
    - name: Push changes
      uses: ad-m/github-push-action@master
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        branch: ${{ steps.extract_branch.outputs.branch }}

[mattduck]

@0ut0fcontrol thanks for sharing this, it's super cool! I'll find somewhere sensible to publicise it - at least link to it from the README. I think I should set it up for gh2md itself.

I'm glad this is useful for you and still working OK. I haven't used it much recently, so I'm not sure if there are any problems or obvious features that would help - feel free to open issues if you do have anything.

[0ut0fcontrol]

@mattduck gh2md works well. If I found any problems or features I will open a issues for it.

My purpose is to backup my repo with issues.

And the issues is readable even when I do not has access to github.com.

Combining gh2md and pandoc --self-contained served my purpose.

(I am not test other tools yet, this may not be the best way to do this.)

Here is a script to setup crontab to fetch repos and convert issues.md to issues.html.

#!/bin/bash

# crontab -e
# 0 1 * * * bash path_to/backup_issues/backup_issues.sh &> path_to/backup_issues/backup_issues.sh.log
# backup_issues
# ├── backup_issues.sh
# ├── repo1
# ├── repo2
# └── repo3

DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
for i in $DIR/*/; do
    date
    cd $i
    pwd
    git fetch --all --verbose
    git pull
    test -f issues.md && pandoc --self-contained issues.md -o issues.html --metadata title=backup_issues
    test -f issues.html && echo "backup issues in ${i}issues.html"
    echo "#############################################################################"
done

[mattduck]

@0ut0fcontrol I've added this to the repo now - worked fine first time. Thanks again!

If I get some time soon, I'll add a flag to optionally remove the timestamp message in the file - this way you'll only get a commit if the content has changed, instead of a new commit on every run due to the unique timestamp.

[0ut0fcontrol]

@mattduck Thank you.

It seems that gh2md doesn't download closed PR.
Is it an expected behavior?

[mattduck]

@0ut0fcontrol I've just fixed that. Back when I wrote this I didn't need to export closed PRs, and for some reason instead of making it configurable I just hardcoded it.

I've updated the default behaviour to fetch everything by default. You can selectively disable parts with --no-closed-prs, --no-closed-issues, --no-prs or --no-issues.

[mattduck]

I'm gonna close this issue just so I know that there's no work to do on it. Will leave it pinned though.

[JimmyLv]

The workflow always has this issue, is that because I am using Chinese?

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)

[0ut0fcontrol]

@JimmyLv Yes, gh2md with python2 can not process issues with Chinese characters.
Using python3 will solve this problem, see 0ut0fcontrol/jimmylv.github.io@76a967c.

I will update the example in this issue and create a PR to modify issues2md.yml in gh2md.

[milahu]

my version of .github/workflows/issues2md.yml (used here)

• use env GITHUB_ACCESS_TOKEN, option --token was removed
• set flag --multiple-files to produce one file per issue
• set flag --idempotent to reduce diff noise
• write output files to archive/github/issues/
• use pandoc to convert github-markdown to strict-markdown
  • github issue: github-markdown
  • github tree: strict-markdown
  • file extension .ghmd is not-yet supported in github tree
  • debian: pandoc 2.5 (old, bugs)
  • nix: pandoc 2.17

# .github/workflows/issues2md.yml
# https://github.com/mattduck/gh2md/issues/11

name: Issues2Markdown
on:
  #push: # comment it to reduce update.
  schedule:
    # every day
    #- cron: "0 0 * * *"
    # every hour
    - cron: "0 * * * *"
jobs:
  build:
    name: Backup github issues to markdown files
    runs-on: ubuntu-latest
    steps:
    - name: Set output path
      run: echo "GH2MD_OUTPUT_PATH=archive/github/issues/" >> $GITHUB_ENV
    - name: Check output path
      run: |
        if ! [[ "$GH2MD_OUTPUT_PATH" =~ ^[a-zA-Z0-9_/.+~-]+$ ]]; then
          echo "error: output path does not match the pattern ^[a-zA-Z0-9_/.+~-]+$"
          exit 1
        fi
    - name: checkout
      uses: actions/checkout@master
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
    - name: Run gh2md
      run: |
        pip3 install --user --upgrade setuptools
        pip3 install --user gh2md
        export PATH="$HOME/.local/bin:$PATH"
        gh2md --version || true
        export GITHUB_ACCESS_TOKEN=${{ secrets.GITHUB_TOKEN }}
        # fix: RuntimeError: Output directory already exists and has files in it
        git rm -rf $GH2MD_OUTPUT_PATH
        # workaround for https://github.com/mattduck/gh2md/pull/31
        mkdir -p $GH2MD_OUTPUT_PATH || true
        gh2md $GITHUB_REPOSITORY $GH2MD_OUTPUT_PATH --idempotent --multiple-files --file-extension .ghmd
        #sudo apt-get install pandoc # pandoc 2.5 == too old
    # install nix to install pandoc 2.17
    - name: install nix
      uses: cachix/install-nix-action@master
      with:
        nix_path: nixpkgs=channel:nixos-unstable
    - name: "pandoc: convert github-markdown to strict-markdown"
      uses: workflow/nix-shell-action@main
      with:
        packages: pandoc
        script: |
          set -x
          pandoc --version || true
          find $GH2MD_OUTPUT_PATH -name '*.ghmd' -type f | while read path
          do
            base="${path%.*}"
            pandoc --verbose -f gfm+hard_line_breaks -t markdown_strict "$base.ghmd" -o "$base.md"
          done
    - name: "cleanup: move .ghmd files to separate folder"
      run: |
        mkdir -p $GH2MD_OUTPUT_PATH/ghmd/
        mv -v $GH2MD_OUTPUT_PATH*.ghmd $GH2MD_OUTPUT_PATH/ghmd/
    - name: Commit files
      run: |
        git add $GH2MD_OUTPUT_PATH
        git config --local user.email "[email protected]"
        git config --local user.name "GitHub Action"
        if ! git commit -m "up $GH2MD_OUTPUT_PATH" -a
        then
          echo nothing to commit
          exit 0
        fi
    - name: Get branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: get_branch
    - name: Push changes
      uses: ad-m/github-push-action@master
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        branch: ${{ steps.get_branch.outputs.branch }}

[Karmaibringkaosanddrama]

Thank you for your great work.

I add my github actions example here, in case someone needs it.

it backups all issues every day.

Because it only backup the issues belong to the repository that contains this workflow. It does not limit by the GitHub API rate.

# gh2md log:
...
Writing to file: issues.md
Github API rate limit: RateLimit(core=Rate(reset=2020-09-17 02:50:05, remaining=977, limit=1000))
Done.

# .github/workflows/issues2md.yml
name: Issues2Markdown
on:
  push: # comment it to reduce update.
  schedule:
    # every day
    - cron: "0 0 * * *"
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
    - name: Backup github issues to a markdown file.
      run: |
        pip3 install --user --upgrade setuptools
        pip3 install --user gh2md
        $HOME/.local/bin/gh2md $GITHUB_REPOSITORY issues.md --token ${{ secrets.GITHUB_TOKEN }}
        git add issues.md
    - name: Commit files
      run: |
        git config --local user.email "[email protected]"
        git config --local user.name "GitHub Action"
        git commit -m "Backup all issues into issues.md" -a
    - name: Extract branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: extract_branch
    - name: Push changes
      uses: ad-m/github-push-action@master
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        branch: ${{ steps.extract_branch.outputs.branch }}