How to Test the Performance of Exalate for Jira Data Center

    This article describes the performance testing of the Exalate App in the Jira Data Center. 

    We've tested the performance and scalability according to testing requirements for Data Center apps in Atlassian Marketplace.

    The results provide an understanding of how the Exalate app impacts a Data Center cluster deployment depending on the configuration.

    We also compared the performance of Jira with the app and without the app to define differences in system performance and performance degradation.

    Testing stages:

    • executed tests using Jira DC without Exalate;
    • executed tests using Jira DC with Exalate using 1 node Data Center deployment (with warm cache) and Jira Server with Exalate.
    • executed tests using Jira DC with Exalate using 2 nodes Data Center deployment (with warm cache) and Jira Server with Exalate.
    • executed tests using Jira DC with Exalate using 4 nodes Data Center deployment (with warm cache) and Jira Server with Exalate.


    We've used the following methods for performance testing:

    • Browser testing (Chrome browser) for performance and scale testing of Jira UI (User Actions) and Exalate UI (User Actions).
      User actions were executed manually and the response time of each request was manually measured via dev tools(Network), collected, and added to the table of results.
    • Automated testing via Postman (Desktop version) for Jira and Exalate API performance and scale testing.
      Created collections of API actions. Each action consists of a request with configs and params required for the execution. The collections were executed via automatic collection run.

    Environment


    Client-side environment:Server-side environment (application)
    Processor

    2.5 GHz Intel Core i7


    Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz

    Installed memory (RAM)16 GB32Gb
    System typemacOS High Sierra Version 10.13.6 (17G65)Ubuntu 16.04.5 LTS - 4.9.124-xxxx-std-ipv6-64


    • The Jira Data Center consists of 4 Jira instances with the Exalate app on each node + 1 Postgres 9.4 (all in one local network #1). Apache2 is used as a load balancer.
    • The Jira Server consists of 1 Jira instance with the Exalate app + 1 Postgres 9.4 (all in one local network #2).
      Jira Data Center and Jira Server communicate via the web with the help of the Exalate app.
      We've used Nginx as a reverse proxy on top of both the Data Center and the Jira Server. All communication requests between the Data Center and Jira Server went through Ngnix.
    • The Jira Data Center database is deployed on the same physical hosts (machines) as the nodes themselves.
    • The Jira Server database is deployed on the same physical hosts (machines) as the nodes themselves.

    Dataset

    We've populated the Jira Data Center with the following set of data


    Data entity typequantity
    1Users4902
    2Projects1501
    3Issues1,000,021
    4Custom fields1408
    5Workflows452
    6Attachments659,994
    7Comments9,903,367


    User Actions with the Exalate App for Jira Data Center

    We scripted the postman to perform a user action from a list of available actions and immediately move on the to next action (i.e. zero think time).

    Each test was a dry run for 20 minutes, after which statistics were collected.


    Jira UI User Actions that result in Exalate code being triggeredExalate API Actions
    1Add WorklogCreate Sync Request
    2

    Edit Work Log

    Create Sync Response
    3Delete Work LogCreate Blob Request
    4Attach FileCreate Blob Response
    5Delete Attached FileCreate Error Response
    6Vote Via ShortcutCreate Blob Error Response
    7Vote By Clicking 'vote'Get Node Info
    8Vote By Clicking 'vote' On The 'View Votes' PageDownload blob
    9Remove Vote Via 'Remove vote for this issue'Poll - Get Sync Requests
    10Watch Issue Via 'Watch issue'Poll - Get Sync Response
    11Watch issue Via 'start watching this issue'Poll - Get Blob Requests
    12Delete WatcherPoll - Get Blob Responses
    13Create Sub-taskPoll - Get Error Responses
    14

    Create Issue

    Poll - Get Blob Error Responses
    15Convert to Sub-taskPoll - Received Sync Request
    16Move issuePoll - Received Sync Response
    17Link IssuesPoll - Received Blob Request
    18Clone IssuePoll - Received Blob Response
    19Edit Label
    20Delete Issue
    21

    Add Comment Via Comment Button


    22

    Edit Comment


    23

    Delete Comment


    24

    Inline Edit Issue


    25

    Edit Issue


    26

    Assign via Assign Button


    27

    Bulk Change: edit Issues


    28

    Bulk Change: move Issues


    29

    Bulk Change: transition issues


    30Bulk Change: delete issues
    31Bulk Change: watch Issues
    32Bulk Change: stop watching Issues
    33Search Issues (by JQL)
    34View Issue


    Test Results from Sept 2019

    Check the diagrams below to see how the Exalate app affects the performance of the Jira Data Center environment.

    Endpoint testing


    Scale testing