Skip to content

Home

Jump to bottom Edit New page
patrick-austin edited this page Apr 26, 2023 · 18 revisions

Purpose

This page is aimed at developers. It does not tell you how to install or use icat.server but how to test any modifications you may make. The build and test relies upon maven - so there is a pom.xml in the top directory. You may choose to develop with an IDE; both Eclipse and NetBeans work with maven however no artefacts from these IDEs are stored in GitHub.

What to do

The development and test procedure requires a dedicated database (MySQL, MariaDB or Oracle) and some dedicated disk space. It is dedicated in the sense that the tests will overwrite the database and the file areas.

Install an authn.db authenticator with the following accounts:

  • notroot / password
  • piOne / piOne
  • piTwo / piTwo
  • root / password
  • guest / guess
  • CIC / password

Install an icat.server (and if 4.9.0 or greater an icat.lucene)

Create or edit ~/.m2/settings.xml to contain a suitably modified version of:

<settings>
   <profiles>
      <profile>
         <activation>
             <activeByDefault>true</activeByDefault>
         </activation>
         <properties>
             <serverUrl>https://smfisher:8181</serverUrl>
             <luceneUrl>https://smfisher:8181</luceneUrl>
             <javax.net.ssl.trustStore>/home/fisher/pf/glassfish4/glassfish/domains/domain1/config/cacerts.jks</javax.net.ssl.trustStore>
             <containerHome>/home/fisher/pf/glassfish4</containerHome>
         </properties>
      </profile>
   </profiles>
</settings>

When you run maven it will attempt to deploy to glassfish. For this to work you must have asadmin in your path and have one domain configured to run on port 4848. It is deployed with --force=true so that it should overwrite anything currently deployed.

Rather than running mvn install you may prefer to run mvn install -DskipTests and then run the tests from your IDE in which case you should ensure that serverUrl, luceneUrl and serverCert are set as system properties. When running from maven, the properties are taken from the ~/.m2/settings.xml as environment variables and passed into the tests as system properties. These settings are not all required for the icat.server - some are needed for the ids.server.

Making a release

First time

To make a release, you will need an account on repo.icatproject.org. Ask Matt Richards to create an account on this machine for you. Put the username and password in a file in the .m2 directory in your home directory (eg. /home/glassfish/.m2/setttings.xml) as described here.

Snapshot release

  • Before (or concurrent with) making a release of ICAT Server, you will have to make a release of ICAT Client. If your changes to ICAT Server would result in changes to the wsdl file, you will have to build and locally deploy a new version of ICAT Client as it will be needed by the integration tests of ICAT Server.
  • Edit the documentation at src/site/xhtml to reflect any changes you have made. Include changes to release-notes.xhtml.
  • Change the version number in the pom.xml - at the top for the ICAT Server version and further down for the ICAT Client dependency version
    • Also change the version in TestWS.java
    • Also change the version in the README
  • run mvn clean install site deploy site:deploy to create the snapshot release.
  • check the documentation at https://repo.icatproject.org/site/icat/server/

Final release

After making and testing the snapshot release, the proper version can be released. Note that for both the following commands, a dry run can be performed by adding the -DdryRun flag. This will run the same steps as above (cleaning, installing etc.) in addition to tagging the version in Github. Arguments can be passed to the underlying commands by using -Darguments=.

  • mvn release:prepare
    • Will require authentication to GitHub
    • Will clean existing files, so ensure the current user has sufficient permissions
    • Will run unit and integration tests, so ensure environment is setup for this and they are passing
    • The version at the top of the pom file should still have the suffix -SNAPSHOT
  • mvn release:perform
    • Displays the next version number when it starts but ignore this.

Vagrant build and test instructions

These instructions follow on from the Vagrant installation tutorial

Overview

Now that the full software stack is installed and running, the next step for a developer is to be able to compile and test the source code. For ICAT server, the authn.db authentication plugin must be installed due to the tests assuming it's existence. These instructions assume that you have followed the Vagrant setup from the installation tutorial and were tested and verified against that assumption, but changing paths and URLs should allow the instructions to be applicable to any environment.

##Installations:

In order to be able to clone the git repository git must be installed.

yum install git

The build and test process uses maven, so it too must be installed.

yum install maven

Install authn.db plugin

NB: commands beyond this point should be run as the glassfish user

Download and unzip

cd ~/downloads
curl -O 'https://repo.icatproject.org/repo/org/icatproject/authn.db/2.0.0/authn.db-2.0.0-distro.zip'
cd ~/install
unzip ~/downloads/authn.db-2.0.0-distro.zip

Configure the authentication plugin

Change directory to authn.db and create the setup.properties file. It has 2 sections. The first is the same as the previous chapters and sets up the server configuration. The second section tells the authentication plugin how to access the MariaDB database. The db.driver parameter specifies the location of the database access code - in the jar file which we copied earlier. The db.url parameter tells ICAT where to contact the database. In this case, it uses the jdbc-mysql connector to access the local machine on port 3306 (the default for MySQL/MariaDB) and uses the icatdb database. Finally, we specify the database username and password for ICAT to use. We created these in Chapter 2.

#Glassfish
secure = true
container = Glassfish
home = /home/glassfish/payara41
port = 4848

# MySQL
db.driver      = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
db.url         = jdbc:mysql://localhost:3306/icatdb
db.username    = icatdbuser
db.password    = icatdbuserpw

For run.properties, you can just copy the run.properties.example file.

Configure verbose logging

Configure DEBUG level logging to aid in troubleshooting.

cp logback.xml.example logback.xml

Change


        <root level="INFO">
                <appender-ref ref="FILE" />
        </root>

to


        <root level="DEBUG">
                <appender-ref ref="FILE" />
        </root>

Also note in this file that the log file location is set to ${HOME}/logs/icat.log.

Install the authentication plugin

Installing most ICAT components consists of running ./setup install. We use the -vv flag here to turn on extra verbosity so that we can see what the setup script is doing.

./setup -vv install

Add DB users

Currently we have installed the authn.db authenticator but we have no user accounts. The tests expect a certain set of accounts, so we need to add these accounts to the database.

Access the MySQL command line tool by running the command

mysql --user=icatdbuser --password=icatdbuserpw

This should then open a MySQL prompt. We need to specify which database we want to access running the command

USE icatdb;

This then allows us to make changes to the icatdb database. PASSWD is the table that contains the login information, and it has two columns - username and password. Running the lines below allows inserts the correct account details needed for the tests.

INSERT INTO PASSWD VALUES ('notroot', 'password');
INSERT INTO PASSWD VALUES ('piOne', 'piOne');
INSERT INTO PASSWD VALUES ('piTwo', 'piTwo');
INSERT INTO PASSWD VALUES ('root', 'password');
INSERT INTO PASSWD VALUES ('guest', 'guess');
INSERT INTO PASSWD VALUES ('CIC','password');

Running SELECT * FROM PASSWD; after will confirm that the new users were added.

To exit the MySQL prompt, enter the EXIT command.

Undeploy

Before we can try building the ICAT server from source, we need to undeploy the icat.server that was set up in the previous steps of the tutorial. We need to undeploy over uninstalling because we need to keep the JDBC connection pool available, which uninistalling removes.

asadmin undeploy icat.server-4.9.1

Maven settings

Create ~/.m2/settings.xml and put following in it, making sure that the urls are correct and that the paths are mapped correctly for your install. The example below should work if you have followed the tutorial instructions.

<settings>
   <profiles>
      <profile>
         <activation>
             <activeByDefault>true</activeByDefault>
         </activation>
         <properties>
             <serverUrl>https://localhost.localdomain:8181</serverUrl>
             <luceneUrl>https://localhost.localdomain:8181</luceneUrl>
             <javax.net.ssl.trustStore>/home/glassfish/payara41/glassfish/domains/domain1/config/cacerts.jks</javax.net.ssl.trustStore>
             <containerHome>/home/glassfish/payara41</containerHome>
         </properties>
      </profile>
   </profiles>
</settings>

Clone repo

We need to actually obtain the source code now. Creating a new code folder seperates the source code from the rest of the files created by the tutorial.

mkdir code
cd ~/code
git clone https://github.com/icatproject/icat.server.git

Modify files

Change to the icat.server folder. Ensure that src/main/resources/run.properties is configured properly. If you are following along with the Vagrant install then the default run.properties should have the correct values.

Change src/main/resources/logback.xml to have debug level

Change


        <root level="OFF">
                <appender-ref ref="FILE" />
        </root>

to


        <root level="DEBUG">
                <appender-ref ref="FILE" />
        </root>

Run tests

Everything should now be set up to be able to build the ICAT server and run the associated tests. mvn install will build, deploy and test the source code.

Developing new integration tests

The integration tests in icat.server rely on a built version of icat.client to interact with itself. This can cause problems if you are trying to test a change which has affected the SOAP WSDL files, as the icat.client will not recognise any new changes. To combat this, you must build icat.client on your local machine pointing at your local ICAT, and then tell icat.server to use this local built version of icat.client. In order to do this, you will need to run the following command to place the icat.client jar in your local maven repository.

mvn install:install-file -Dfile="C:\path\to\icat.client\target\icat.client-4.10.2-SNAPSHOT.jar" -DgroupId=org.icatproject -DartifactId=icat.client -Dpackaging=jar -Dversion=4.10.2-SNAPSHOT

Then, you will need to edit your pom.xml file in icat.server to point to this new "snapshot" version of icat.client (in this case 4.10.2-SNAPSHOT). This will then enable you to develop unit tests successfully, and when you release icat.server just make a corresponding icat.client release as well. The CI may fail temporarily though.

Add a custom footer
Add a custom sidebar
Clone this wiki locally