Skip to content

mkolesnik/ovirt-engine

BranchesTags
 
 

Repository files navigation

Developer mode installation

Preparations

Prerequisites

Install the following system components:

  • java-11-openjdk-devel

  • java-1.8.0-openjdk-headless

  • mime-types or mailcap

  • unzip

  • openssl

  • bind-utils

  • postgresql-server >= 10.0 / rh-postgresql10-postgresql-server

  • postgresql >= 10.0 / rh-postgresql10-postgresql-server

  • postgresql-contrib >= 10.0 / rh-postgresql10-postgresql-contrib

  • python-dateutil / dateutil

  • python-m2crypto / m2crypto

  • python-psycopg2 / psycopg

  • python-jinja2 / Jinja2

  • libxml2-python / libxml2[python]

  • python-daemon

  • otopi >= 1.1.0

  • ovirt-setup-lib

  • maven-3 >= 3.5

  • ansible >= 2.7.2

  • ansible-runner-service-dev >= 1.0.1

  • ovirt-ansible-roles >= 1.1.6

  • ovirt-engine-metrics

  • python-ovirt-engine-sdk4

  • ansible-lint / python3-ansible-lint

  • python-flake8 / pyflakes (optional)

  • python-pep8 / pep8 (optional)

  • python-docker-py (optional)

  • python2-isort (optional)

Note on Java versions

The project is built and run using java 11. 4.3 branches and earlier are excluded. The only exception is GWT code, which still needs to be compiled using jdk-8, and that it because the gwt version doesn’t support the language syntax, and solid support for it is not imminent. That’s why jdk-8 is needed only for build-time , and only for GWT, specifically the javac command.

Prepare your dev envrionment for java 11

  • Use alternatives command to configure java and javac to version 11:

$ sudo alternatives --config java

There are 4 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           java-latest-openjdk.x86_64 (/usr/lib/jvm/java-12-openjdk-12.0.1.12-1.rolling.fc30.x86_64/bin/java)
   2           java (/opt/jdk-9/bin/java)
   3           java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.3.7-5.fc30.x86_64/bin/java)
*+ 4           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc30.x86_64/jre/bin/java)

Enter to keep the current selection[+], or type selection number: 3

$ sudo alternatives --config javac

There are 2 programs which provide 'javac'.

  Selection    Command
-----------------------------------------------
*+ 1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc30.x86_64/bin/javac)
   2           java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.3.7-5.fc30.x86_64/bin/javac)

Enter to keep the current selection[+], or type selection number: 2

  • Maven >= 3.5 is required, download and extract if not installed using distribution package management, and add to PATH.

Fedora 30

$ sudo dnf module install maven

  • verify your mvn version:

$ mvn -v | head -1
Apache Maven 3.5.4 (Red Hat 3.5.4-5)

  • export JAVA_HOME if mvn is not executing using java-11:

#put this in your ~/.bashrc
$ export JAVA_HOME=/lib/jvm/java-11

$ mvn -v | grep "Java version: "
Java version: 11.0.4, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-11-openjdk-11.0.4.11-0.fc30.x86_64

WildFly 15 is required along with ovirt-engine-wildfly-overlay. Preferred way is to install following packages:

  • ovirt-engine-wildfly

  • ovirt-engine-wildfly-overlay

Both packages can be installed from ovirt-master-snapshot-static repository:

[ovirt-master-snapshot-static]
name=Latest oVirt master additional nightly snapshot
baseurl=http://resources.ovirt.org/pub/ovirt-master-snapshot-static/rpm/@DIST@$releasever/
enabled=1
skip_if_unavailable=1
gpgcheck=0

Please replace @DIST@ with fc for Fedora or el for Centos/RHEL.

Alternatively, repository list can be updated using the following command:

$ sudo yum install -y http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm

OVN/OVS is an optional dependency. If you want to use it, check the requirements in the ovirt-engine.spec.in file for a list of packages. Otherwise, you should reply 'No' when asked about it by engine-setup.

For Fedora prerequisites installation, following command can be applied:

$ sudo yum install -y $(cat automation/check-patch.packages)

Install additional packages for Fedora:

$ sudo yum install -y \
      ansible \
      ansible-runner-service-dev \
      bind-utils \
      libxml2-python \
      m2crypto \
      mailcap \
      openssl \
      ovirt-ansible-roles \
      ovirt-engine-metrics \
      ovirt-engine-wildfly \
      ovirt-engine-wildfly-overlay \
      ovirt-setup-lib \
      python2-daemon \
      python2-dateutil \
      python2-jinja2 \
      python-ovirt-engine-sdk4 \
      unzip

Optional packages for Fedora:

$ sudo yum install -y \
      python2-docker-py \
      python2-flake8 \
      python2-isort \
      python2-pep8

System settings

Build locales requires at least 10240 file descriptors, create the following file, replace <user> with user that is used for building, and logout/login:

/etc/security/limits.d/10-nofile.conf
<user> hard nofile 10240
#<user> soft nofile 10240  # optional, to apply automatically

If soft limit was not set, before building, apply new limit using:

$ ulimit -n 10240

Development environment by default uses ports 8080 (HTTP), 8443 (HTTPS) and 8787 (java debug), so make sure they are accessible from the outside. For example, on fedora:

firewall-cmd --add-port=8080/tcp --permanent
firewall-cmd --add-port=8443/tcp --permanent
firewall-cmd --add-port=8787/tcp --permanent

If you also want to connect to the database from the outside:

firewall-cmd --add-port=5432/tcp --permanent

Finally, apply changes using:

firewall-cmd --reload

If compiling in a virtual machine, javac might experience difficulties on guests with dynamically growing RAM so it’s better to have VM’s starting allocation and maximum allocation set to the same value.

PostgreSQL accessibility

Initialize PostgreSQL configuration files:

$ sudo postgresql-setup --initdb --unit postgresql # fedora

Configure PostgreSQL to accept user and password:

Locate pg_hba.conf within your distribution, common locations are:

  • /var/lib/pgsql/data/pg_hba.conf

  • /etc/postgresql-*/pg_hba.conf

  • /etc/postgresql/*/main/pg_hba.conf

  • /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf (el7 with rh-postgresql10)

Within pg_hba.conf set method to password for 127.0.0.1/32 and ::1/128 for IPv4 and IPv6 local connections correspondingly.

If you want to make postgres accessible from the outside, change 127.0.0.1/32 to 0.0.0.0/0 and ::1/128 to ::/0.

Tune PostgreSQL configuration: Locate postgresql.conf within your distribution, common locations are:

  • /var/lib/pgsql/data

  • /etc/postgresql*

  • /var/opt/rh/rh-postgresql10/lib/pgsql/data/postgresql.conf (el7 with rh-postgresql10)

Within postgresql.conf make sure following values are set:

max_connections = 150
work_mem = 8MB
autovacuum_max_workers = 6
autovacuum_vacuum_scale_factor = 0.01
autovacuum_analyze_scale_factor = 0.075
maintenance_work_mem = 64MB

If you want to connect from the outside, set also:

listen_addresses = '*'

Enable and start (systemctl enable rh-postgresql10-postgresql --now) or restart the PostgreSQL service (systemctl restart rh-postgresql10-postgresql for the SCL one).

Database creation

Create database for ovirt-engine, usually the following sequence should work to create a user named engine that owns database named engine:

# su - postgres -c "psql -d template1" # if on fedora or
# su - postgres -c "scl enable rh-postgresql10 -- psql -d template1" # if on el7 with rh-postgresql10
template1=# create user engine password 'engine';
template1=# drop database engine;
template1=# create database engine owner engine template template0
encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
template1=# \q

Enable uuid-ossp extension for the database:

# su - postgres -c "psql -d engine" # if on fedora or
# su - postgres -c "scl enable rh-postgresql10 -- psql -d engine" # if on el7 with rh-postgresql10
engine=# CREATE EXTENSION "uuid-ossp";
engine=# \q

Ansible Runner Service configration

Since oVirt 4.4 the engine is integrated with Ansible Runner Service. To properly integrate the development environment with the Ansible Runner Service, you need to edit /etc/ansible-runner-service/config.yaml file as follows:

playbooks_root_dir: '$PREFIX/share/ovirt-engine/ansible-runner-service-project'
ssh_private_key: '$PREFIX/etc/pki/ovirt-engine/keys/engine_id_rsa'
port: 50001
target_user: root

Where $PREFIX is the prefix of your development environment, which you’ve specified during the compilation of the engine.

After edditing the file, make sure you’ve restarted the Ansible Runner Service service:

# systemctl restart ansible-runner-service
# systemctl enable ansible-runner-service

Development

Environment

Development environment is supported only under non-root account. Do not run this sequence as root.

Each instance of application must be installed at different PREFIX and use its own database. Throughout this document application is installed using PREFIX="${PREFIX}" and engine database and user, these should be changed if a new instance is required. Do not mix different versions of product with same PREFIX/database.

From this point on, the "${PREFIX}" will be used to mark the prefix in which you selected to install the development environment.

Build

To build and install ovirt-engine at your home folder under ovirt-engine directory execute the following command:

$ make clean install-dev PREFIX="${PREFIX}"
Note
 ${PREFIX} should be replaced with the location in which you intend to install the environment.
Note
 Add DEV_BUILD_SCL_POSTGRESQL=1 in order to configure your local build for SCL rh-postgresql10.
Note
 Add SKIP_CHECKS=1 to disable tests.

Build targets

all

Build project.

clean

Clean project.

all-dev

Build project for development.

install-dev

Install a development environment at PREFIX.

dist

Create source tarball out of git repository.

maven

Force execution of maven.

generated-files

Create file from templates (.in files).

When creating new templates, generated files will be automatically appears in .gitignore, updated .gitignore should be part of commiting new templates.

==== Build customization

The following Makefile environment variables are available for build customization:

PREFIX

Installation root directory. Default is /usr/local.

BUILD_GWT

Build GWT. Default is 1.

BUILD_ALL_USER_AGENTS

Build GWT applications for all supported browsers. Default is 0.

BUILD_LOCALES

Build GWT applications for all supported locales. default is 0.

BUILD_DEV

Add extra development flags. Usually this should not be used directly, as the all-dev sets this. Default is 0.

BUILD_UT

Perform unit tests during build. Default is 0.

BUILD_JAVA_OPTS_MAVEN

Maven JVM options. Can be defined as environment variable. Default is empty.

BUILD_JAVA_OPTS_GWT

GWT compiler and dev mode JVM options. Can be defined as environment variable. default is empty.

Note
 Note that BUILD_JAVA_OPTS_GWT overrides BUILD_JAVA_OPTS_MAVEN when building GWT applications (BUILD_JAVA_OPTS_MAVEN settings still apply, unless overridden).
DEV_BUILD_GWT_DRAFT

Build "draft" version of GWT applications without optimizations. This is useful when profiling compiled applications in web browser. Default value is 0.

Following changes are applied for draft builds: - Prevent code and CSS obfuscation. - Reduce the level of code optimizations.

+ On local development environment, using GWT Super Dev Mode (see below) is preferred, as it automatically disables all optimizations and allows you to recompile the GWT application on the fly.

DEV_BUILD_SCL_POSTGRESQL

Configure your local development deployment to be used with SCL rh-postgresql10 instead of system PostgreSQL. Default value is 0.

DEV_BUILD_GWT_SUPER_DEV_MODE

Allows debugging GWT applications via Super Dev Mode, using web browser’s JavaScript development tooling. Default value is 0.

Do a local Engine development build as you normally would. Then, start the Super Dev Mode code server as following:

$ make gwt-debug DEV_BUILD_GWT_SUPER_DEV_MODE=1

In your browser, open http://127.0.0.1:9876/ and save the "Dev Mode On" bookmark. Next, visit the GWT application URL (as served from Engine) and click "Dev Mode On". This allows you to recompile and reload the GWT application, reflecting any changes you’ve made in the UI code.

DEV_EXTRA_BUILD_FLAGS

Any maven build flags required for building.

For example, if your machine is low on memory, limit maximum simultaneous GWT permutation worker threads:

DEV_EXTRA_BUILD_FLAGS="-Dgwt.compiler.localWorkers=1"
DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS

Any maven build flags required for building GWT applications.

By default, GWT applications are built for Firefox only. To build for additional browsers, provide comma-separated list of user agents, see frontend/webadmin/modules/pom.xml for full list.

For example, to build for Firefox and Chrome:

DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=gecko1_8,safari"

To build for all supported browsers, use BUILD_ALL_USER_AGENTS=1.

For example, to build only the English and Japanese locale:

DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.locale=en_US,ja_JP"

To build for all supported locales, use BUILD_LOCALES=1.

For example to build engine without obfuscated Javascript code:

DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.style=pretty"

To build engine without obfuscated CSS styles:

DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.cssResourceStyle=pretty"
DEV_REBUILD

Disable if only packaging components were modified. Default is 1.

PY_VERSION

Python defaults to python3 if available, use PY_VERSION=2 in order to override.
This options affects various services and several features written in python.

Note
 engine-setup which runs otopi, uses different customized variable OTOPI_PYTHON
WILDFLY_OVERLAY_MODULES

Change location of WildFly overlay modules. If you want to disable WildFly overlay configuration completely, please set to empty string. Default is /usr/share/ovirt-engine-wildfly-overlay/modules.

Setup

To setup the product use the following command:

$ "${PREFIX}/bin/engine-setup"
Note
 [1] If on el7, make sure you run scl enable rh-postgresql10 bash first.
[2] otopi, and therefore engine-setup, now defaults to python3 except el7, use:
$ OTOPI_PYTHON=/usr/bin/python2 "${PREFIX}/bin/engine-setup"
to override.

During engine setup, a certificate has to be issued and you will be asked for a hostname. If you want to use imageio-proxy along with the engine, it has to be the name by which your machine is accessible from the outside.

JBoss

If you want to use different WildFly/EAP installation, specify it at --jboss-home= parameter of setup.

Environment

OVIRT_ENGINE_JAVA_HOME

Select a specific Java home.

OVIRT_ENGINE_JAVA_HOME_FORCE

Set to non zero to bypass Java compatibility check.

Refresh

If there are no significant changes, such as file structure or database schema, there is no need to run the setup again, make install-dev <args> will overwrite files as required, run engine-setup to refresh database schema.

Do remember to restart the engine service.

If there is a significant change, safest path is to stop service, remove ${PREFIX} directory, build and setup.

The ${PREFIX}/bin/engine-cleanup tool is also available to cleanup the environment, it is useful for application changes, less for packaging changes.

Service administration

Most utilities and services are operational, including PKI, host deploy.

To start/stop the engine service use:

$ "${PREFIX}/share/ovirt-engine/services/ovirt-engine/ovirt-engine.py" start

While the service is running, this command will not exit. Press <Ctrl>-C to stop service.

Access using HTTP or HTTPS:

Remote debug

By default, debug address is 127.0.0.1:8787. If you want to make engine accessible to the remote debugger, after running engine-setup edit the following file: ${PREFIX}/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf:

ENGINE_DEBUG_ADDRESS=0.0.0.0:8787

Running instance management (JMX)

ovirt-engine service supports jmx as management interface. Actually, this is the standard jboss jmx interface, while authentication can be done using any engine user with SuperUser role. Access is permitted only from the local host.

Access JMX shell using provide OPTIONAL_COMMAND for non interactive usage:

$ "${JBOSS_HOME}/bin/jboss-cli.sh" \
  --connect \
  --timeout=30000 \
  --controller=localhost:8706 \
  --user=admin@internal \
  --commands="OPTIONAL_COMMA_SEPARATED_COMMANDS"

Useful commands:

Modify log level
/subsystem=logging/logger=org.ovirt.engine.core.bll:write-attribute(name=level,value=DEBUG)
Create a new log category
/subsystem=logging/logger=org.ovirt.engine:add
Get the engine data-source statistics
ls /subsystem=datasources/data-source=ENGINEDataSource/statistics=jdbc/
Get threading info
ls /core-service=platform-mbean/type=threading/

By default JMX access is available only to localhost, to open JMX to world, add ${PREFIX}/etc/ovirt-engine/engine.conf.d/20-setup-jmx-debug.conf with:

ENGINE_JMX_INTERFACE=public

GWT debug

$ make install-dev PREFIX="${PREFIX}"
$ make gwt-debug

Debug port is 8000, detailed instructions for GWT debugging are here.

GWT debug URL, provided components running on same machine:

Note that gwt.codesvr parameter does not apply when using Super Dev Mode.

DAO tests

Create empty database for DAO tests refer to Database creation.

Provided user is engine, password is engine and database is engine_dao_tests.

$ PGPASSWORD=engine \
  ./packaging/dbscripts/schema.sh \
    -c apply -u engine -d engine_dao_tests

Run build as:

$ make maven BUILD_GWT=0 BUILD_UT=1 EXTRA_BUILD_FLAGS="-P enable-dao-tests \
  -D engine.db.username=engine \
  -D engine.db.password=engine \
  -D engine.db.url=jdbc:postgresql://localhost/engine_dao_tests"

VM console

After the environment is setup and installed, some adjustments are required.

Copy vmconsole-host configuration:

$ sudo cp -p "${PREFIX}/share/ovirt-engine/conf/ovirt-vmconsole-proxy.conf \
/etc/ovirt-vmconsole/ovirt-vmconsole-proxy/conf.d/50-ovirt-vmconsole-proxy.conf

If selinux is enabled on your machine, set type on vmconsole helper using:

$ sudo chcon --type=bin_t "${PREFIX}/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py"

RPM packaging

$ make dist
$ rpmbuild -ts @tarball@
# yum-builddep @srpm@
# rpmbuild -tb @tarball@

The following spec file variables are available for package customization:

ovirt_build_quick

Quick build, best for syntax checks. Default is 0.

ovirt_build_minimal

Build minimal Firefox only package. Default is 0.

ovirt_build_gwt

Build GWT components. Default is 1.

ovirt_build_all_user_agents

Build GWT components for all supported browsers. Default is 1.

ovirt_build_locales

Build GWT components for all supported locales. Default is 1.

Example:

# rpmbuild -D"ovirt_build_minimal 1" -tb @tarball@

About

This is a mirror for http://gerrit.ovirt.org, for issues use http://bugzilla.redhat.com

Resources

Readme

License

View license
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

366 tags

Packages

No packages published

Languages

  • Java 85.2%
  • CSS 8.2%
  • PLpgSQL 2.4%
  • Python 2.1%
  • JavaScript 1.1%
  • Shell 0.5%
  • Other 0.5%