Integrated Simulation Databases

Since the late 1990s, ExtendSim has had an embedded database as part of its simulation tool. Now in its second generation, it is so incredibly useful that I can’t imagine building a model without it. In this post, I’ll describe my favorite advantages of using the internal ExtendSim database but for a more comprehensive description of the major features please read ExtendSim Advanced Technology: Integrated Simulation Database (Diamond, Krahl, Nastasi, and Tag 2010).

Here is a list of some of the major features of the ExtendSim database:

• Rapid data access
• Connectivity to other data sources
• Parent \ child relationships
• Support for easy component scaling
• Multiple databases in one model
• Database aware modeling components
• Database address attributes
• Embedded distributions
• Excel Add-in
• Data linking
• Link alerts

The first thing you should know about the ExtendSim database is that speed consideration was (and still is) a very high priority in its design. I have seen cases where a modeler used Excel or Access as the primary simulation data repository in such a way that the model interacted with it constantly during the model run. Having that constant interaction with Excel or Access during a model run tends to really slow the model down. Interacting with the internal ExtendSim database during the model run is comparable to the speed of interacting with arrays, which is really fast.

Using the internal database does not prevent you from connecting to other data repositories like Excel or Access or ODBC or ADO; however, the best practice when using another data repository is to import the input data into the ExtendSim database ONCE at the beginning of the model run, interact with the ExtendSim database during the run and then export the results to the external data repository ONCE at the end of the model run. This gives you the ability to use an external data repository to store your input and output data while using the ExtendSim database for speed during the model run.

The next major benefit of using the ExtendSim database is its visibility and the separation of model and data. I once had a discussion with someone who had just completed building a model – and he didn’t use a database! His model had roughly 300 processes in it. He used best guesses for the process times, because he didn’t have the actual data at the time he was building the model. When I spoke to him, he was beginning to get real data, and he wanted to start testing the sensitivity of the model. He was having a difficult time with the task. All of his process data was hidden in the block dialogs, which were spread out across the model. It was difficult for him to see the data he was currently using without going into every single process and looking for it!

Understandably, he was frustrated. He asked me if there was an easier way, and I suggested that he update his model using the ExtendSim database. The database helps make all of your data visible in organized table structures. This enables you to use best guesses as you are building the model, and afterwards, you can easily find and make modifications when you get real data. Using the database, the original best guess data will not get lost and forgotten.

The embedded ExtendSim database also allows the user to create parent \ child relationships between tables. This has a number of advantages.  First and foremost it helps endure data integrity but it also helps with making the data readable so the user does not have to maintain separate lists of indexes.     

The embedded ExtendSim database can also be used to help scale a model. Often, models have many processes that are similar, if not identical, except for their data. In ExtendSim, constructs like this can be encapsulated in a hierarchical block (h-block). Encapsulating the construct into an h-block makes duplication of these similar processes much easier, and it helps organize the model. In order to make maintenance of the encapsulated construct easier, the h-block can be placed in a library so that if changes are needed within the constructs, modifications may be made in one h-block and those same modifications can be automatically replicated to other identical h-blocks. The difficulty comes when the constructs have slightly different input data. This can be handled easily though by using the database to store the input data instead of storing it in the block dialogs.

Let me show you what I mean.  In the illustration below, the station construct is stored in an h-block in a library. Each station has a different process time and capacity. The process time is stored in a database table in which each station looks at a different record. Each station construct is reading the database for that process time, so each process time can be unique, even though the h-block construct for the four stations are identical. The only difference really is that they read from different records for their process time and capacity.

In summary, using the internal ExtendSim database can help make your simulation data visible and easy to find. It can speed up the run time, compared to constantly interacting with an external data repository during the simulation run. It is much easier to work with the data when the data repository is native to the simulation tool, and it can be used to help scale your model as it grows. Keep in mind that this is just a short list of the key benefits; there are many others.

If you have not already started using the ExtendSim database, I would highly encourage you to check it out.  I also teach a week long class on the ExtendSim database. We spend about two days covering the mechanics of using the database and about three days on the techniques of how to integrate it throughout a model. I cover almost all of the tricks I know. So if you have the time, then I encourage you to come and learn how to effectively use the ExtendSim database.  

YouTube video

I also have included a 30 minute overview of the ExtendSim Database on youtube.  Check that out when you have a chance.

https://youtu.be/1TnNUKzoqtw

I have included some references below for further reading.  The first one, ExtendSim Advanced Technology: Integrated Simulation Database, is a more in-depth look at the advantages of the ExtendSim database.  The next two references are some good examples of users taking full advantage of the ExtendSim database by not just creating a model but creating their own application within ExtendSim.

Diamond, B., Krahl, D., Nastasi, A., Tag, P., 2010. ExtendSim Advanced Technology: Integrated Simulation Database. In Proceedings of the 2010 Winter Simulation Conference, eds. B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yucesan, 32-39. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Saylor, S., Dailey, J., 2010. Advanced Logistics Analysis Capabilities Environment. In Proceedings of the 2010 Winter Simulation Conference, eds. B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yucesan, 2138-2149. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Akiya, N., Bury, S., 2011. Generic Framework for Simulating Networks using Rule-Based Queue and Resource-Task. In Proceedings of the 2011 Winter Simulation Conference, eds. S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu, 2194-2205. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Benefits of Simulation

I recently had a discussion with a scheduler at a manufacturing plant. I was trying to explain to him the benefits of simulation, and I thought it might be a good discussion to post here as well.

First, here is a brief, although certainly not all-inclusive, list of the key benefits of simulation:

• Answer Questions
• Identify Problems
• Animation & Visualization
• Communication
• Understanding

I am sure you are well aware of these benefits and others. But I want to take a moment and focus on the one I think is most important – Understanding. There are two aspects of this I want to elaborate on.

First, the process of modeling provides the model builder an in-depth look at the existing system. When someone takes the time to build a simulation model, it is much more involved than building a flowchart. A simulation model should be a close representation of the system. The results of the model must validate with reality, which is something you can’t do with a flowchart. The act of building a representative model drives the modeler to understand the flow, the routing logic, the priorities, the resource constraints, and the decisions made on the plant floor!  The modeler must spend time understanding the day to day operational decisions made on the shop floor which goes far beyond the effort one would ever put into a flowchart or a value stream map.

The second aspect is more interesting. Many system experts might have an understanding of their existing system on the surface, but they don’t understand the “physics” of it. Let me explain. The scheduler I mentioned earlier completely understood his current factory. However, if the scheduler were to change the rules he applied, and/or change the job priority scheme at various processes, and/or reduce or increase the batch size, and/or reduce or increase the total Work In Process, and/or change any other major aspect of the system – then neither the scheduler nor anyone else in the plant would be able to predict the effects!

However, think of what would happen if you gave the plant scheduler a simulation model. He could experiment with the model and fairly accurately predict what would happen with policy changes like the ones previously mentioned.  With this toolset, the scheduler could experiment with various policy changes to see what would affect the system in the most positive way and could radically improve the system.  Using simulation, all of the disruptive testing of these policies would be done in the model and NOT on the plant floor. The plant scheduler could really become the Yoda of the plant.

Embracing Simulation

Over the last 10 years, more than a thousand students have attended one or more of my simulation classes. I have seen successful and not so successful students. There are ways to help ensure success and here I provide some guidance on avoiding the most common mistakes.

Simulation modeling can be challenging for a beginner.  There is a sharp learning curve not only on the mechanics and nuances of the software, but also with managing a simulation project.  However, there is a magic formula and here it is in a nutshell. Make sure you have…

• Training on the software
• Support from a mentor
• An appropriate first project
• Support from the rest of the company
• Access to data
• Subject matter experts to educate yourself on the process
• Time to work on the project
• Time to verify and validate the project
• Time to measure its success

Training & Mentoring

Get training on the tool as well as a mentor - both are crucial.  The cost of a one-week training class will save two to three months of learning and struggling on your own.  I also would recommend spending some time with a modeling mentor.  If your company has other modelers, buddy up with one of the experts for a couple of models.  If no one else uses simulation at your company, you might want to consider hiring someone to help build the first model.  After the first project, consider hiring that consultant again as a mentor through the next project.  It is difficult for a new modeler to build a good model without knowing what a good model looks like.  I have seen modelers using simulation for years but using the same bad habits they originally started out with because they have never seen it done differently.  So get started out on the right track.

Successful Initial Projects

The first project should be selected with care.  It should be used to help you get comfortable with simulation.  The purpose of your FIRST simulation project should NOT be to create a model that automagically generates the optimal schedule for the factory!  Although that is a lofty goal, first pick a simple project, possibly a subset of what will eventually become part of a larger model.  Start out small.  Pick something that will be successful.  Learn how to manage a simulation project.  Don’t be afraid to make mistakes.  Yes, make mistakes!  We actually learn the most when we make mistakes and correct ourselves. 

Support

Get support from the rest of the company.  A simulation model needs data, often lots of it.  A simulation model needs process knowledge.  As a new modeler, you often do not have either of those within your immediate grasp.  You must go find the information from others within the company.  This information will take resources that you often can’t commandeer yourself.  It takes a certain level of management support to commandeer those resources.  Without this support, you can’t get data plus you can’t get the information you need regarding the process.  So without the right level of management support, the project can’t succeed.

Time

Give yourself time to work on the project.  If you have other responsibilities, quite often your simulation responsibility will fall to the bottom of the task list.  I have seen this way too often.  When a new modeler has other responsibilities and is always starting and stopping frequently; it can be detrimental!  Each time a modeler revisits the model they tend to have forgotten the last few things they had done and waste a lot of time getting back into the groove.  This has a similar effect with experienced modelers, but not to the same extent as a new modeler.

Verification & Validation

Budget plenty of time to verify and validate the model.  A computer will do what you tell it to do, which is not necessarily what you want it to do. While you are building the model, constantly take time to verify that the model is doing what you intended. As you complete stages of the model, validate that it is a reasonable representation of the real system. For validation, you will need the same subject matter experts that helped you understand the process. This step will not only help to ensure that the model is accurate, it will make others more confident in the model.

Measuring Success

This is not easy, but if you can, compare a decision made without the model to one made with the model. Compare the performance of the implemented system with the model. If there are any significant differences, make note of them so that they can be accounted for in the future. If the model proved to be accurate, use this as political capital for future projects.

Here is a link to a paper I co-authored with Dave Krahl (now at Kromite LLC) a few years ago on getting started with simulation.   It is a great read and contains more details. Dave also helped write this post as well, thanks Dave.

http://www.informs-sim.org/wsc11papers/131.pdf

Don’t Pay with Pennies

Have you ever been in line behind someone who pays with pennies? It’s not good is it? There’s time for counting, higher chance of error, and not enough space in the cash register to store all of the pennies. However, I see the same technique in simulation models quite often. This is because modelers tend to make a literal translation of the system into the simulation model. With a little thought, it is often more efficient to work with in terms of a group rather than a single item.

Let’s look at some examples of paying with pennies in simulation models:

• Processing a batch of parts one-at-a-time. Typically this is breaking the batch into individual units processing them and re-batching them into a single unit for transport to the next operation.
• In Logistics we WANT to model supply chains where every case represents an item.  We WANT to see every case / pallet get moved around and individually placed on the truck for the shipment.  The model should really only care about the fact that it took X minutes to load the truck and X, Y, and Z amounts of inventory was removed from the warehouse.
• Allocating an array one row at a time. If you are storing results in an output table or an array it is tempting to add results one row at a time as you generate the results. However, this will cause the simulation software to allocate memory in small chunks. And, memory allocation is a relatively slow process in the simulation model. I do know that if you allocate one row in an ExtendSim database table, space is reserved for more rows should you need them, but this is still much less efficient than adding all of the rows that you will need once at the start of the simulation.

When building a model, look for opportunities to group actions together. Consider the work that the CPU will have to go through to simulate the model that you built. When you can:

• Group items together and process them as a single unit. Use math and attributes to calculate model features such as delays and yield rates.
• Track information in tables instead of individual items. For example inventory can be represented by a series of linked database tables.
• Perform operations once, at the start of the simulation. This works well for setting up arrays and tables.
• Use discrete-rate simulation to model high speed processes such as bottling and filling lines.

While I use ExtendSim, the above techniques would be useful for any simulation software. Considering methods for reducing the number and type of calculations will yield benefits both in modeling and run time.

I would like to thank Robin Clark for his comments and input on this topic.