2.6.1

Dependencies updates

Upgrades: Play 2.8.6, Scala 2.12.13, SBT 1.4.7, commons CSV 1.8

FEATUREs

• exclude all access by IP, see doc. https://github.com/jmvanel/semantic_forms/blob/master/doc/en/administration.md#customize-sf-site
• #204 - Completion in "display URI or search" field
  /search : new parameter link= , completion of dbPedia
  • now 3 criteria for searching: text, class, and link
  • "link" is like backlinks, so not necessarily a dbPedia URI
  • all 3 are facultative
• #209 admin mode for /create , /save , /edit , /load-uri, /load, /update
  • /login , /authenticate , /register, /logout are not subjected to filter by mode policy, because admin should be able to log,
• /tools is now accessible to unlogged users
• /search, /display now accept MIME text/csv with Turtle headers
  consistent with https://ontology.uis-speleo.org/howto/#Csv
  see doc. for the command line tool:
  https://github.com/jmvanel/semantic_forms/wiki/Semantization#convert-csv-in-rdf
• Paragraphs & history view: filter out rdf.typ in form syntax
  • reduces output, already grouped by rdf.typ
• RDF Cache Algo:
  • for ontologies in /, load into relevant graph
    • Case of many ontologies ( foaf: )
      FIXES the duplications that existed since the origins
      Now we only create a single graph named after the ontology
    • exploit triples like:
      foaf:Document rdfs:isDefinedBy <http://xmlns.com/foaf/0.1/>.
      which refer to ontology.
  • an URI with fragment is loaded into an graph named without the fragment
    Case of many ontologies ( dcat: )
    FIXES the duplications that existed since the origins: RDF URLs u1#f1, u1#f2, etc.
    used to create as many named graphs with identical content, for example:
    http://www.w3.org/ns/dcat#Dataset
    http://www.w3.org/ns/dcat#theme
    Now we only create a single graph named .
    It reduces network traffic, and above all it makes things easier in the event of an ontology update.
• possibility to override head.html, tail.html with local files at runtime
• No login mode, in forms_play/conf/nologin.routes , without possibility to edit content
  • cf thread https://gitter.im/playframework/playframework?at=5ffd9e5ce578cf1e95e2e3c9
  • run with
    authbind --deep bin/semantic_forms_play -J-Xmx300M -J-server -Dhttp.port=80 -Dhttps.port=443 -Dplay.http.router=nologin.Routes
• Italian localization; need review by native Italian speakers!

FIXes

• old known bug: only user herself can modify password
  • to modify password, use /login , being already logged
• retrieveURIResourceStatus and hence updateLocalVersion were called twice
• RDFCacheAlgo updateLocalVersion: avoid systematic reload for bad HTTP headers
  • if Expired is older than LastModified, do not take it in account
  • if LastModified is in the future, do not take it in account
    TESTED with void:Dataset whose HTTP headers are incoherent
    http://localhost:9000/create?uri=void%3ADataset&prefill=no

Source code is one commit ahead at tag https://github.com/jmvanel/semantic_forms/releases/tag/2.6.1-csv-fix-url

Group user edits by RDF class, Jena 3.17.0 , Play 2.8.5

Lots of enhancements and fixes, like saving RDF datatypes when saving forms, RDF lists display, HTTP timeouts, multi-values in table cells, etc

Landing page: group user edits by RDF class ("paragraphs & history view")
FEATURE: Proxy for HTTP get; used for HTTP images that need to be HTTPS; same for dbPedia Lookup
FEATURE: Create Prefilled input Form, from the Referer URL,when user click on (+) button
FEATURE: IP Filter to blacklist unwanted web crawlers; for now hard coded IP

I18N : spanish, german

Blank Nodes as subject: implement in /download
/sparql-form , /display?displayuri=...&blanknode=true , download?url=_%3Abaaf... , /sparql-form?query=

/download : use prefixes actually used among the global SF Prefix Mapping
add new service /rdf2json?src=<URL>&frame=
Form <input> : paste text with new lines  creates new input fields in same order
dbpedia lookup: adapt to new API
add new service /json2rdf?src=<URL> (WIP)
Use new taxref HTTPS sparql server for taxon name completion
New client program: RDFuploader: loads given file in given graph;
contextual creation button (+) : clone (alias prefill) occurs only for (new) button on top of /display page
TOC generation (scrollnav)
FIX CSVImporter when cell contains a Turtle abbreviated URI
FIX /login : user ID was an URI, now a string,

Dependencies updates
Jena 3.17.0 , Titanium JSON-LD 1.0.0 ,
com.typesafe.play 2.8.5 , sbt.version=1.4.5 , Scala 2.12.12,
summernote 0.8.18

The zip distribution was updated for better logs and performance of /load-uri , if you want that more recent code, the tag is https://github.com/jmvanel/semantic_forms/releases/tag/2.6.0-log .

Integration of tools Sparklis, SPOggy and LodLive

This is already in production, but marked pre-release, expecting other users' tests.

BUG FIX

• continuation for /search & /backlinks
• forms: input of dates
• FIX importing a SF URI into another SF server (HTTP HEAD for /ldp : Content-type should be json-ld, was text/plain)

FEATURES

• report HTTP (HEAD) error status of Internet URI's
• consistently hide experts buttons under a ... button
• prevent a FORM SUBMIT with ENTER
• redefine a label in a specific form, different from the one for the property in ontology
• site site-specific customization: local file head-extra.html allows to add markup in HTML <head>
• add SPOggy and LodLive diagram tools
• geo-map: Leaflet: zoom in further stretching tiles: use case map a garden or park
• use Link HTTP header rel='alternate' for declaring other RDF formats, like dbpedia.org does
• /table page now accepts HTTP parameter unionDefaultGraph=on like /sparql, /sparql-ui and /select-ui
• Sparklis, the SPARQL requester , now works with SF

Dependencies updates

com.typesafe.play 2.6.19, sbt.version=1.1.6 , Jena 3.8.0 , log4j, scalaz-core, play-json, lucene-suggest, scalajs-jquery, scalajs-dom, scalatestplus-play

SPARQL POST fix

BUG FIX

• SPARQL Select POST
• SPARQL nows return HTTP 500 in case of error
• URI's with non ASCII letters (IRI's) can be created by forms
• a PDF URL value in a triple caused a stack trace in web page /display

FEATURES

• History view for SELECT queries in /tools page : sparql= parameter added to /history - NOTE the query only filters locally generated content, the Internet content in cache is not shown
• checkbox for unionDefaultGraph in UI pages for SPARQL Select and Construct
• /sparql GET service redirects to SPARQL UI page for a browser
• Add link "Named graph" towards /showTriplesInGraph?uri= in page /display, if it is not a locally hosted URI
• Lucene : index sioc:content
• Add completion on RDF class in search form, like for instance creation

The release actually corresponds to this commit :
964a465

Bug & performance fixes

• Upgrade dependencies: Jena 3.7.0 , scala-xml, play-json, apache-any23-csvutils, log4j, lucene-demo, lucene-suggest
• Selenium Scala DSL tests

BUG FIX

• FIX slow /create form (from 18s to 3s on a database with many ontologies)
• /load : FIX HTTP 413 "Request EntityToo Large : limit is set to 8Mb
• re-Activate completion for URI fields in form, and in general search button
• Recover from any exception, for all Web Pages services & /sparql-form
• When Loading an URL, remove HTML fragment (anchor) ; that can make the Jena call to fail
  cf Jena issue https://issues.apache.org/jira/browse/JENA-1467
• FIX exception during saving form: case of HTML created in SummerNote
• FIX issue when saving big content HTML strings containing possibly Turtle, etc
• In /backlinks, there was extra line feeds
• FIX /backlinks button in the case of an URI with #
• FIX importing a SF URI from one instance into another (HEAD did not declare default Content-Type)
• FIX ; so SF pages look better in Slack
• FIX issue : no correct login forms from an empty instance
• FIX reading a .n3 URL

Performance

• change URIs for forms specifications that are not self-hosted
  NOTES
  • it was a cause of slowing displaying of forms,
    as SF tried to download stuff that did not exist,
    and since there is a timeout on the download, it slows down the loading of a page
  • now the bootstrap of an empty SF is easier, even when not having passed populateRDFCache;
    it is enough to follow links which download the forms and ontologies

FEATURES

UI

• geo-map:
  • use navigator language to filter rdfs:label's ;
  • image in popup fom foaf:img or foaf:depiction
• GPS forms: add altitude
• I18N
  • take in account all user languages gotten from HTTP header when filtering triples
  • button "Show all" when there are triples in non-user languages
• Continuation Form (OFFSET, LIMIT) works for /search /history, and /backlinks
• Tables: alternate line color with CSS
• simplify columns in /history
• Move site message up
• modified_by and other less important stuff are now with small font (CSS class)
• Class creation input now uses completion for rdfs:Class (and hence owl:Class)
• Add (+) button for creation in /display page for a class,
• Completion and search: take again in account super-classes : performance issue FIXED !! by using UnionGraph, thanks to Andy's advice
• "expert buttons" are now hidden; there is a button "..." to display them
• /backlinks now loads given URL in the cache , suggestion from Marco !
• geographical Map Button in /search and /backlinks pages
• Display rdf:type in /backlinks & /search (only hyperlink and icon )

BACKEND

• /load : also accept RDF/XML and JSON-LD
• enhance LDP: listing LDP containers. Total tests run: 90, Failures: 24, Skips: 31 => Passed: 35
• new service /load-uri?uri= , like /display but only loads into RDF cache, no HTML page creation

Buttons for geo. map table view, GPS enhanced

BUG FIX

• HTTPS URL's are needed for summernote css/js 8c4f3f6
• Library rdf-ext-all-1.0.0.src.js is loaded in HTTPS like all the rest (thanks bergos!)
• trim URI String From User
  • replace spaces with _ in phone numbers, remove spaces in emails
• fix Sort button in /backlinks page
• When using both HTTP and HTTPS on same server, the Map link in /backlinks was making mixed HTTP and HTTPS content
• dbPedia lookup: do not set QueryClass, as it works often not
• dbpedia lookup: set timeout: 30000

Performance

• Play! is now serving HTML results in gzip
  • => quicker page loading, especially on slow networks

FEATURES

UI

• button for geo. map in /backlinks page a752a5e
• button for /table view on each URI in the form fd3a328
• Continue to update positions in form after first position received from GPS
  • FIX in GPS JS code: new PositionOptions() does not work at runtime
• When annotating a plain HTML page, pre-fill rdfs:label with string after last /
• do not show the draggable anchor if there is no HTML id (in /backlinks page)
• Use case: a non-URI string was in database, but an URI is entered by user
• Table View: add edit button in first column, subject URI

Security

Do not log password in clear !!!!!!!!!

BACKEND

• Load RDF URL's: when HTTP header gives no content-type, try URL extension 2c32570

Zip reloaded after commit 7cb7b6a (FIXEs for editing complex texts + Button for geo. map in /backlinks page )

FIX Lucene search, geo. maps, HTML editor

UPGRADES

• jenaVersion = "3.6.0"
• upgrade any23-csvutils to latest "2.1"
• Upgrade Scala.js "0.6.21" , scalajs-dom "0.9.4 (latest)

BUG FIX

• FIX /lookup and /search
  • Lucene indexing: deactivate "graph-specific indexing" (search was taking 1 minute for hundreds of graphs)
• SPARQL services : recover a memory exception without the application stopping

FEATURES

UI

• HTML inline editor in SF forms (button "EDIT"): use summernote.org

• when <property> form:shortString true , the button "EDIT" is not shown
  used for login form, FOAF

• New page for geographic map with LeafLet & OSM from any SPARQL query, parsing JSON-LD with RDF-ext JS library

  • accessible from /tools page
  • compute the bounding rectangle in JavaScript

• geo:lat, long are only filled from GPS if input Is Empty

  • set GPS Parameters in JS call to watchPosition() : will need some tuning later maybe ...

• button (+) near every class to create instance of that class

• page /showTriplesInGraph?uri= : Show graph size

• Forbid usernames of length <= 1

• page /select-ui does not demand logging

Services

• New HTTP parameter in /sparql : enrich: activates addition to results of rdfs:label from TDB
  used in Geo maps
• /sparql-data is now also a GET service

CLIENTS

• SPARQLquery2SFcacheApp.scala : SPARQL query to Semantic_Forms cache, as an independant client:
  • From a SPARQL query to another endpoint,
    load URI's returned by the query into SF
  • in coherence with essential feature of SF: LOD (semantic) cache

Traceability on a triple for user and timestamp, FIXes

A few things in this release. The secondary RDF database for history of user's editions is now fully leveraged in displaying an URI.

• FIX #183 : now traceability on a triple for user and timestamp is complete: different triples for the same subject and property can have different user and timestamp.
• Use case: "load an RDF document into a user's graph, and be able to edit the content"
• Add bold warning when RDF document's URL is not a subject inside itself
  "No triple for this URI! Click on subjects link above."
• FIX /load service with graph= specified
• UI
  • Do not show "Draw Graph Link" for non-Internet URI's
  • Do not show "normal Navigation Button" for non-Internet URI's

New services /load, /table, /sparql2; multi-type, search counting links

UPGRADES:

• Jena 3.5.0
• Play! framework 2.5.18
• Scala.js 0.6.18

FEATURES:
Services

• New POST service /load
• New service /table making a table from a SPARQL, which can hold any list of enriched triples
• new service /sparql2 with union graph, with POST/GET, SELECT/CONSTRUCT
  all graphs are considered at the root level
• HTML (and RDF) Content negotiation on local URI's
• recover a memory exception without the application stopping

UI

• completion in form for internal data by /lookup service similar to dbPedia lookup
  ( the possible URI's by type are no more in the HTML page, which greatly reduces page size)
• browser can remember user and password
• multi-type: an URI can have several types that will contribute to the form
• many usability improvements
  • login
  • Fold by default view create area
• modified by ... is now a hyperlink to user account (user traceability: show author and timestamp of each triple)
• When URI is a class, new icon (+) to "Create an instance" (of that class)
• search loads an URL if it sees one
• Download The current URI in the 3 RDF syntaxes
• new view path length 1 or 2 + historical order allows to display communities centered on a URI. This is related to extended search
• search counting links similar to the Google rank
• new form inference owl:sameAs
• form view improvements
  • fields (triples) are editable based on: 1) the current user being the original author 2) triples from Internet are immutable, but it's possible to add new values from subject + property
  • filling geographic coordinates for the geo:long and geo:lat properties (from the HTML5 geolocation API)
• other views improvements
  • 90f02ca /history (and /index) shows rdf:type as a new "Type" column
  • fbc7320 thumbnail icons in /history, /wordsearch
  • 49e5059 Link to full history in landing page /index
• New trait GeoController: display table of distances travelled by each mobile

ARCHITECTURE

• Implement & test new generic controller /page?feature=
• new interface ServiceListener, general callback for HTTP requests (used for RDF links count)
• new interface deductions.runtime.utils.SaveListener (used for RDF links count)

MISC:

• login and register account (sign-in): replace Play! forms with semantic_forms forms
• Add icons (foaf:img) to classes foaf:Person, foaf:Project, foaf:Document, owl:*
• Cornichon test for /ldp
• Cornichon test for /form-data
• Adding script to move a given subject (or all subjects) in a graph to another graph
• Adding the lang of the request on each new triple
• Add prefix wikidata == wd ; add wikidata:P18 to imageProperties

REFACTOR:

• Move val's to def functions in Form2HTMLDisplay
• Moving each field on FormSyntax + changing all rawDataForForm by formSyntax
• Delete trait ApplicationFacade
• split all "routes functions" in tree different files :
  • WebPages --> for all route used by the generic application
  • Services --> for all web service
  • SparqlServices --> for all service implementing SPARQL standards
• Cleaning import in packages html, sparql_cache, forms_play, utils, forms and connector
• changing the package for Configuration.scala and DefaultConfiguration.scala
• Delete ApplicationFacadeJena

BUG FIX:

• FIX update_server.sh : version must be coherent with build.sbt
• FIX exception in /display when in logout mode, application deployed from zip archive
• OWLsameAsFormProcessing: FIX SPARQL: each part comes from a different graph - for "Automatically show inferred values by RDFS+ inference"
• FIX doc/fr/letsenscript.html (and applied it) ; enhance scripts/rupdate.sh
• Residual bug on completion with + button
• FIX login form and display User in mailto: case
• FIX saving a form with /save service after /create: case when several values for 1 property

INCOMPATIBILITIES

• a small incompatibility in the service /form-data: the field "type" is now a JS array:
  http://semantic-forms.cc:9112/form-data?displayuri=http%3A%2F%2Fsemantic-forms.cc%3A9112%2Fldp%2FJean_Delahousse
  • Otherwise, no software incompatibility.
• there have been updates on Lucene, so the index must be rebuilt (it takes a minute via deductions.runtime.jena.lucene.TextIndexerRDF)

Traceability of user edits, enhance CSS, etc...

FEATURES

• ADDING user traceability for each triple in edit and display mode
• fill Geographic Coordinates on page load, for properties geo:long and geo:lat (from HTML5 geolocation API)
• /display : show RDF syntax errors of downloaded RDF documents
• Improve CSV import: Remove graph before populate it
• Add script forms_play/dist/scripts/rupdate.sh : remote SPARQL update with wget
• login name in top right in most pages
• Display in form header link to the form specification or "automatic form"

UPGRADE:

• Upgrade Jena to 3.3.0: search problems are solved
• Upgrade SBT to 0.13.15
• Upgrade Scala.js to latest version

MISC.

• Add dbPedia completion for skos:broader skos:narrower skos:related
• streamline RDF database transactions: reduce code in transactions to a minimum
• REFACTOR:
  • extract package connectors from forms/ into new SBT module connectors/
  • extract Scala source from package utils in forms/ into new SBT module utils/
• SBT: GREAT MOVE to real Multi-project build
• Add new SBT project contacts_management, for "Modularisation en applications classiques (contacts) #150
• ADD selenium test on Tool page
• ADD cornichon test for /sparql, /display and /form-data

BUG FIX:

• FIX HTTPS
• FIX links in history view, in DashboardHistoryUserActions
• FIXes: SPARQL error in getETagFromDataset(); wrap In Transaction updateLocalVersion()
• FIX editing a Locally Managed URI, in getLocallyManagedUrlAndData
• FIX FEATURE: annotate plain Web site: a transaction was missing , in pureHTMLwebPageAnnotateAsDocument()
• FIX bug when adding widget

UI:

• CSS: REFACTOR and FIXES: colors, named graphs, tools page
• ADD button to show and hide the header part : display, search, creation