Skip to content

Latest commit

 

History

History
108 lines (87 loc) · 3.62 KB

README.md

File metadata and controls

108 lines (87 loc) · 3.62 KB
Raw

Line processing utility using Javascript as a proof of concept

Requires QuickJS for compilation

Compile

cd src
qjsc.sh -o qjslp qjslp.js

Usage

qjslp -h
Line processing utility using Javascript (0.1.0)
Usage: qjslp [-h|--help] [-c,--code] [-b,--begin] [-e|--end] [-q|--quiet] [-d|--debug] [--debug-store]
    -c  --code:     code to execute for each line ($_ is the current line, $$ is the global store object,
                    $i is the 0-based index of current line)
                    Can be specified multiple times
    -b, --begin:    code to execute before processing lines ($$ is the global store object)
    -e, --end:      code to execute after all lines have been processed ($$ is the global store object,
                    $c is the number of lines which were processed)
    -q, --quiet:    if used, input lines will never be printed
    -d, --debug:    print debug information during processing (implies -q)
    --debug-store:  print store whenever a store value has changed (ignored if -d is not set)
    -h, --help:     print help

Inside a code section

  • if function returns undefined, null or false line will be ignored
  • if function returns something which is not a string or a number, input line will keep its value
  • if $_ is assigned a new value, input line will have this new value

Following function are provided as helper in begin, end and code sections

  • $p : print to stdout
  • $e : print to stderr

Following shortcuts are also provided in code sections

  • $_!.{fn} => $_ = $_.{fn} (example: $_!.toUpperCase() => $_ = $_.toUpperCase())

Examples

NB : a better alternative is likely to exist using common unix tools ;)

  • print the number of entries for each state in file 01.csv, in JSON format
    • initialize counter object in store
    • for each line
      • ignore if line is first one (header)
      • store current state in store
      • initialize counter for current state if needed
      • increase counter for current state
    • print counter object
cat 01.csv | qjslp -q \
    -b '$$.count = {}' \
    -c 'return $i > 0' \
    -c '$$.state = $_.split(",")[9].trim()' \
    -c '$$.count[$$.state] ||= 0' \
    -c '++$$.count[$$.state]' \
    -e '$p(JSON.stringify($$.count))'
  • convert 01.csv to JSON format
    • initialize entries array in store
    • for each line
      • remove "
      • if line is first one (header), split line and store result in store
      • split line, build a JS object and add it to the entries array
    • print entries object
cat 01.csv | qjslp -q \
    -b '$$.entries = []' \
    -c '$_!.replace(/"/g, "")' \
    -c 'if ($i == 0) { $$.fields = $_.split(",").map(e => e.trim()) ; return false }' \
    -c 'entry = {} ; $_.split(",").forEach((e, i) => entry[$$.fields[i]] = e.trim()) ; $$.entries.push(entry)' \
    -e '$p(JSON.stringify($$.entries))'
  • extract server section from 02.ini
    • enable ignore flag in store
    • for each line
      • if [server] section is starting, disable ignore flag
      • if another section is starting, enable ignore flag
cat 02.ini | qjslp \
    -b '$$.ignore = true' \
    -c 'if ($_.startsWith("[")) { $$.ignore = ($_ != "[server]") } ; return !$$.ignore'
  • split lines in 03.csv based on sex
    • for each line
      • ignore if line is first one
      • if second field contains F, write line to stdout
      • otherwise write line to stderr
cat 03.csv | qjslp -q \
    -c 'return $i > 0' \
    -c '$_.split(",")[1].trim() == "\"F\"" ? $p($_) : $e($_)' \
    >/tmp/F.csv 2>/tmp/M.csv