Development Resources

Architecture Diagram

Frontend:

  • React - A JavaScript library used to build UI components. We use a combination of functional and class components within our application. Some functional components utilize React hooks like useEffect and useState.
  • Redux - A JavaScript library used to manage application state. We also use redux-forms throughout our application.
  • Material UI - A React component library that we utilize for easing UI design.
  • Plotly - A JavaScript graphing library that we use for data visualizations.
  • Auth0 - DISABLED BY DEFAULT - SaaS which allows for integrating authentication and authorization into our application. We use JWT authentication, as well as assign roles and permissions to users in higher environments. Additonally, we have an SPA + API architecture.

Backend:

  • Node / Express - We use Express as our server framework.
  • PostgreSQL - An object-relational database that we use for data storage.
  • Sequelize - A promise-based JavaScript library which serves as our object-relational mapper.
  • Redis - A data structure store where jobs await execution by BG workers.
  • Bull - A queue used to store queued jobs in Redis.
  • Elasticsearch - A search engine used to index data stored in our database. Currently, we support searches on Tag Names, Accused, and Complainant Names.

Infrastructure:

  • Docker - PaaS which enables us to execute our code in containers. We utilize multiple docker images including ubuntu, elasticsearch, and localstack.
  • Terraform - IaC which enables us to manage varying services and resources within our application.
  • AWS API Gateway - DISABLED BY DEFAULT - Higher environments send requests from client browser through an API Gateway running on an HTTP/2 protocol.

Document Storage

  • LocalStack - An AWS mocking framework used when AWS services are disabled in development.
  • AWS S3 - DISABLED BY DEFAULT - Used for document storage of case attachments and letters.

Logging and Monitoring

Continuous Integration and Deployment

Additionally, feel free to take a look at some of our Architectural Decisions here.

React-Redux Data Flow Diagram

API Documentation

Postman

Postman is a REST client, API runner, and documentation tool. We use it to test out our API endpoints without the overhead of the frontend. You can explore some of the documented endpoints here.

Setting collection up locally:

  • If you want to explore the endpoints locally, you must download Postman first. Once it’s installed, you will need to import the collection as well as the local environment. The files used are located in the <project root>/doc/postman directory. If you already have Postman installed, you can simply click the following: Run in Postman

At the time of writing, they include:

  • Police Data Management.*.json
  • Local.*.json
  • CI.*.json
  • Staging.*.json

Development Patterns

Soft Delete

We use the paranoid flag in Sequelize to ‘soft delete’ records instead of removing them from the database. This is important to know when you run any raw queries that you need to select rows where deleted_at is null, or you’ll pull in data that is supposed to be deleted. This comes into play in our export case information functionality.

Wrapping Migrations in a Single Transaction

When we write migrations, we wrap the changes in “transactions” which are essentially promises that Sequelize creates to touch the database. To ensure that we don’t have changes left hanging when a migration fails, we want to wrap the entire migration in a single transaction. There may be many smaller transactions within the single wrapper transaction, but they would all roll back if one fails.

To do so, pass transaction as a param in referenced async functions. Take a look at transformDuplicateCaseNoteActions.js and 20200108192000-remove-duplicates-case-note-actions.js for reference.

Testing

Unit Tests

A unit test is meant to verify that a piece of code (i.e. method, component, process) is functioning as expected. Unit tests are meant to test these units independent of dependencies, so we utilize mocking frameworks like Jest and Nock

  • We split our unit test suites into the following categories:

    • Client - unit tests for client using Jest, Enzyme, and React-Testing library
    • Server & Worker - unit tests for server using Jest, request tests with SuperTest, handler tests with Nock

    Some of these tests utilize Builders defined within src/sharedTestHelpers/ which are helper functions to build out dummy data for our tests.

    Additionally, for some of these unit tests, we use Jest snapshots to verify that components or documents render as we expect.

E2E Tests

End-to-end tests are meant to test the application from the end user’s perspective by simulating a real user session.

  • We use NightWatch to write our automated E2E tests.
    • Sessions are configured to run on Chrome browser using chromedriver.

Integration Tests

An integration test verifies that all modules within a codebase work together as expected.

Health Check

We have a basic health check endpoint to ensure that our server is working and available.

Conditionally Run Tests

You might notice that some test suites have the message “Skipping test, Auth is disabled.”, this is because we have added flags to our application which disable Cloud Services. The benefit of this is that you do not have to have an AWS or Auth0 account to begin using the Complaint Manager app (simply run a docker-compose up app!). However, if you want to, for whatever reason, use authentication or AWS locally, then follow these simple instructions (You will need a login under a tenant of Auth0 and an AWS account to use the app locally):

  • Note: You cannot have a situation in which authentication is on and AWS is off or vice versa. Both authentication and AWS have to be on or both authentication and AWS have to be off.

  1. Locate docker-compose.yml. In this file, you should see:

    services:
    app:
    ...
    environment:
    ...
    - USE_CLOUD_SERVICES=false
    - REACT_APP_USE_CLOUD_SERVICES=false

  2. Now, you must change USE_CLOUD_SERVICES and REACT_APP_USE_CLOUD_SERVICES from false to true.

  3. Additionally, with these services enabled, the application will now expect you to have the following local environment variables:

    • TEST_USER
    • TEST_PASS
    • AWS_ACCESS_KEY_ID (this account should have access to services PDM requires like S3 and SecretsManager)
    • AWS_SECRET_ACCESS_KEY (this account should have access to services PDM requires like S3 and SecretsManager)
  4. Repeat steps 1 and 2 but now for the worker and app-e2e services (within the same file, below the app service).
    • For E2E tests, you must also change variable disableAuthentication in the nightwatch.conf.js file in your instance-files repository from true to false.
    • Note: These changes will now invoke authentication when running yarn e2e locally.
  5. Run up localhost in your terminal like you normally would (docker-compose up app) and the app should now need AWS and authentication credentials and you can move about the app freely.

For instructions on how to execute the tests discussed here, see README

Security

  • We regularly scan for outdated dependencies with Hawkeye Scanner, which runs on all commits. When a vulnerability is identified, we use Snyk Vulnerability Database for information and remediation. Hawkeye is also useful for ensuring that no secrets or passwords are checked into our application code.
  • We subscribe to security advisories for different technologies within our tech stack, so we can receive prompt notification of any incidents or outages (i.e, Docker, AWS, Heroku, etc.).
  • We implement a strict Content Security Policy (CSP) for all our environments using Helmet, which we evaluate using Google’s CSP Evaluator tool.
  • We use SSL for encryption of request/response. SSL certificates are automatically managed and renewed by Heroku.
  • Our Postgres database is encrypted at rest (provided by hosting on Heroku).
  • [Only applicable if enabled] AWS S3 files are encrypted with AES-256 Server-Side Encryption using Amazon S3-Managed Keys (SSE-S3) at rest and in transit.