Projects:Advanced Warehouse Operations/Performance Testing
Contents |
Introduction
The goal of this testing has been to ensure the proper behavior of the application while having an acceptable performance in environments with high volumes of data.
In order to do so, a mix of dummy data and properly created transactional data has been generated to populate the instance.
The results of this test should be used as an approximation to a real environment. However, since the bulk of the data is composed of dummy data, the real performance of the application can slightly differ from the obtained results.
Scenario
The scenario created for the performance testings starts with several Warehouses properly configured to work with Advanced Warehouse Operations.
In this case, a total of eight Warehouses have been created. This number is rather small, but since all the Advanced Warehouse Operations are focused in the same Warehouse, increasing the number of Warehouses had no effect on the performance, therefore it was not needed to create any more.
However, there are some other tables that can have a big impact in Advanced Warehouse Operations if not managed properly. For those tables, this are the number of records that have been created (find here the script for this creation):
Storage Bins
The goal was to have one Warehouse with more than 10 million of Storage Bins and another one, used in most of the performance testing, with more than 100.000 Storage Bins.
The Warehouses with the largest amount of Storage Bins are
| Warehouse | No. of Records |
| Massive Data Warehouse | 11.100.012 |
| US West Coast | 112.307 |
| US East Coast | 11.254 |
Storage Bin Capacity
The goal was to have one Warehouse with more than 10 million of Storage Bin Capacities and another one, used in most of the performance testing, with more than 100.000 Storage Bin Capacities.
In order to do so, a Storage Bin with 10 capacity records has been cloned to achive this figures
The Warehouses with the largest amount of Storage Bins are
| Warehouse | No. of Records |
| Massive Data Warehouse | 10.000.010 |
| US West Coast | 104.088 |
| US East Coast | 18.801 |
Storage Details
The goal was to have one Warehouse with more than 10 million of Storage Details and another one, used in most of the performance testing, with more than 100.000 Storage Details.
The Warehouses with the largest amount of Storage Details are
| Warehouse | No. of Records |
| Massive Data Warehouse | 10.000.002 |
| US West Coast | 111.148 |
| US East Coast | 20.193 |
Tasks
The goal was to have a total of more than 50 million of Tasks, in the same Warehouse that is used for most of the performance testing
The Warehouses with the largest amount of Tasks are
| Warehouse | No. of Records |
| US West Coast | 50.014.565 |
| US East Coast | 21.362 |
Products and Storage Bin Group List
The goal was to have a total of more than 1 million of Products, with their own Storage Bin Group List with 10 lines each, for a total of more than 10 million Storage Bin Group List Lines
| Table | No. of Records |
| Product | 1.021.171 |
| Storage Bin Group List | 1.017.464 |
| Storage Bin Group List Lines | 10.029.831 |
System Specs
As the last part of the scenario, this are the specifications of the System in which the performance testing has been done. This might serve as an starting point to extrapolate results to different machines:
- CPU: Intel Core i7-5500U 4x2.40GHZ
- Memmory: 15.4 GB
- OS: Ubuntu 16.04 LTS 64-bit
- Disk: 838.4 GB
Results
In order to test the performance with the previous scenario, the already existing automated tests have been launched several times to simulate a real life scenario in which all the Warehouse Operations are executed mixed between them.
The central piece of code that manages the Warehouse Operations is the Central Broker, it is called for both Tasks generation and Tasks confirmation.
From the last 174 calls to the Central Broker this were the results
- Number of calls: 174
- Maximum amount of time taken: 16,673 seconds
- Minimum amount of time taken: 0,027 seconds
- Average amount of time taken: 0,986 seconds
In an scenario with a Purchase Order with 1.000 lines that needs to create the Put-Away:
- Without Verbosity Log Enabled: 102,45 seconds
- With Verbosity Log Enabled at Debug Level: 652,797 seconds
Chart of the ime taken in the execution of two runs of the automated tests:
Time taken in the execution of a run of the automated tests in the Console (some tests failed due to wrong data, that was expected):
Time taken in the execution of a run of the automated tests in the Profiler:
Slower queries in the execution of a run of the automated tests:
Considerations over the results
- The time taken in the queries is acceptable considering the size of the tables it has to query
- The average time taken in the calls to the Central Broker is acceptable too
- There are several times when the time taken in the Central Broker is above average. This is due to the number of records it has to loop to find the necessary Storage Bin with the given configuration
- Warehouse Configuration Check and Increment Task Priority Processes are slower than Tasks generation/confirmation
Conclusion
The time taken to generate the Tasks is defined by the configuration (number of records that the process needs to loop) rather than by bottlenecks in the process.
This means that if the Warehouse is configured properly, the performance of the application will be acceptable. If the performance is not what it was expected, the Warehouse Configuration should be reviewed first.
Limit scenarios
Some scenarios can reach the performance limits of Advanced Warehouse Operations.
Put-Away
If the Put-Away Warehouse Algorithm needs to loop over tens of thousands of Storage Bins to find one the performance will be compromised.
This can only happen if the Warehouse itself has tens of thousands of Storage Bins and the Algorithm loops through them all without being able to place the Stock.
This points to a possible Configuration problem
- For 10.000 Storage Bins, in the tests, it has taken around 30 seconds.
- For 100.000 Storage Bins, in the tests, it has taken around 2 minutes.
Both scenarios for a Order with one line
Picking
If the Picking Algorithm needs to loop through thousands of records without finding any Stock, the performance will suffer.
This can only happen if the same Product is placed over thousands of Storage Bins, with very a small quantity in each one of them, and an Order is placed requiring a huge quantity of this Product.
This case points to a possible problem in the Warehouse Configuration.
- The time taken to loop 1.000 records was 1 minute
- The time taken to loop 10.000 records was around 10-15 minutes
Recalculate Occupancy
It works with acceptable times (under 5 minutes) for Warehouses with tens of thousands of Storage Bins.
However, it has big performance issues with Warehouses of millions of Storage Bins.
Warehouse Configuration Check
This process is meant to be used while configuring a new Warehouse, not with one that is already being used.
Also, it works with acceptable times (under 10 minutes) for Warehouses with tens of thousands of Storage Bins.
However, it has big performance issues with Warehouses with millions of Storage Bins.
Additional Information
List of significant tables with more than 1.000 records
| Table | No. of Records |
| obawo_task | 50.035.948 |
| obawo_verbosity_log | 50.027.547 |
| m_locator | 11.226.681 |
| m_storage_detail | 10.134.629 |
| obawo_locator_capacity | 10.122.900 |
| m_attributesetinstance | 10.106.695 |
| obawo_locator_group_line | 10.029.831 |
| m_product_acct | 1.021.215 |
| m_product | 1.021.171 |
| obawo_locator_group_list | 1.017.464 |
| m_transaction | 61.295 |
| m_inoutline | 43.385 |
| m_productprice | 42.310 |
| ad_pinstance | 33.176 |
| m_inout | 29.947 |
| c_orderlinetax | 24.548 |
| c_orderline | 24.539 |
| obawo_batchoftasks | 24.143 |
| fact_acct | 23.330 |
| obawo_locator_group | 17.243 |
| m_stock_proposed | 15.552 |
| fin_payment_schedule | 12.864 |
| fin_payment_scheduledetail | 12.076 |
| c_ordertax | 11.372 |
| c_order | 11.364 |
| obawo_warehouse_rule_assi | 9.582 |
| m_product_aum | 9.047 |
| obawo_receptionlist | 8.977 |
| m_movementline | 8.259 |
| m_movement | 8.259 |
| c_invoicelinetax | 6.123 |
| c_invoiceline | 6.121 |
| m_transaction_cost | 5.676 |
| m_product_org | 3.691 |
| obawo_invprop_line | 1.990 |
| obawo_inventorylist | 1.988 |
| obawo_inventory_prop | 1.988 |
| obawo_internal_routing_assi | 1.928 |
| m_attributeset | 1.917 |
| m_attribute | 1.911 |
| m_costing | 1.836 |
| m_inventoryline | 1.667 |
| fin_payment_detail | 1.518 |
| c_invoice | 1.517 |
| fin_payment | 1.384 |
| m_inventory | 1.313 |
| obawo_pickinglist | 1.220 |
| m_reservation_stock | 1.205 |