The VALCRI architecture is built from different Docker containers that speak with each other using mostly REST interfaces. The architecture is designed to incorporating Security, Ethics, Privacy and Legal (SEPL) solutions. The data stores – the Unstructured Database (UDB) and the Structured database (SDB) – used are controlled by SEPL Enforcement components and a Template Engine manages the previously checked and accepted query templates that can be sent to the data stores. The Advanced User Interface (AUI) server is also designed with SEPL in mind: a Jetty (Java HTTP server and Java Servlet container) instance is created per user by a Jetty Lifecycle Management component. Each such instance lives inside a Docker container to promote isolation. The Jetty instance hosts the mid-tier services, serves the front-end JavaScript code and manages the communication between the mid-tier services and the front-end. The midtier services are written in java and front-end components are implemented using GWT – Google Web Toolkit. The Model View Presenter (MVP) design pattern is implemented. The SEPL Enforcement components, as well as the analysis components, and the Jetty Containers communicate to the CAS (Central Authentication Service) component to ensure that only authorized users can perform certain actions and allows data to be properly restricted. The user management of the CAS component is linked with the LDAP (Lightweight Directory Access Protocol) component to manage and authenticate user credentials. All the components may use the interface exposed by the GrayLog component, ensuring that any action done by any component can be logged in the logging storage. Systematic system integration is a key principle for the success of VALCRI and for that a development pipeline was designed. This pipeline aims to provide continuous system integration while promoting collaboration, contributions, quick feedback to contributions, changing and evolving interfaces, and above all respecting the principle of “keep it working” – allowing to introduce many contributions in small steps while the system continues to compile and work. The GIT Source Control Management (SCM) platform is used for keeping track of changes in the source code. It is accompanied by a GitLab installation that provides a user interface that allows managing the user accounts and the individual code repositories. In order to compile and build the code available from the SCM, a custom build system was developed using Gradle. The Gradle setup is also accompanied with a Nexus component – an artefact repository – which hosts all the compiled, binary VALCRI system components. For each change in the SCM, the source code is automatically rebuilt and all tests are run. In order to do this, a Jenkins installation was put into place. Because the VALCRI architecture includes many services working on different environments, Docker was selected to allow building, shipping and running the complete environments.
Keywords
Visual Analytics, Sense-Making, Criminal Intelligence Analysis, Architecture, REST, Security, Ethics, Privacy and Legal (SEPL), Model View Presenter, Isolation, Google Web Toolkit (GWT), Errai, Docker containers, Central Authentication Service (CAS), LDAP, GrayLog, GIT, Gradle, Nexus, Jenkins
VALCRI WHITE PAPER SERIES