Debugging Tips

I have a few debugging tips that I thought I would share.  If you have other debugging tips then please share them with us.

Test as you go

Don’t wait to the end of model development to start testing.  Test as you build.  If you wait until the end to verify the model works correctly, it will take much longer to find bugs because you have to search through the entire model. Build the model in small chunks and test thoroughly as you go.  If you test as you go, the new bugs will be in the constructs you have just built.  You will be focused on a smaller construct to look for the bug while that logic is fresh on your mind.  This is such a crucial aspect of the modeling process that it deserves mentioning more than once! 

Snipping Tool or Snagit or Print Screen

Get access to a screen capture tool and keep it handy.  When an error message pops up, take a screenshot of it.  Don’t be click happy and absent-mindedly click the OK button when errors appear.  If you don’t document what the errors are, you will have to rerun the model again because you will quickly forget the exact details of the error message.  I think the Snipping Tool is installed with Windows, but I am a big fan of SnagIt.

Logging exceptions

No one can or should constantly look for every potential bad event each time they run the model.  For example, if an item stays in the system 10x longer than you expect, there might be a potential problem.  Or if item type x ever goes through a specific construct then you know you have a potential problem.  You can and should continue to monitor that event in case another bug shows up.  There are many more instances that require monitoring, but they shouldn’t consume our attention. 

These are events where you need to set up a construct to log the “exception” events to an error log table in the database when they occur.  You should also report the number of these exceptions on the main model worksheet when and if they occur, so they are sure to grab your attention.  Once you have finished verifying and validating your model, then you can take the exceptions out or have a flag that turns them off. You may also want to leave them in just to reduce your worry.  Logging error events or exception events is a great way to monitor the system.  Note, however, that if you put error events in that are really warnings, you might overlook real errors when they occur.

 Find dialog

If you have an error message and it refers to an object number, you can use the menu shortcut "Control + F" to open the Find dialog in order to find the object number. 

Global Random Seed

If you need to debug your model and want a repeatable result from run to run, you can fix the Global Random Seed.  Debugging can be difficult if the bug appears at different simulation times from run to run.  Fixing the global random seed can be done in the Random Number tab of the Simulation Setup dialog.  If you want to repeat the last model run, set the Random Seed to the "Starting seed used in the last model run."   Setting the seed to any positive number will allow the model to run with a repeatable stream of random numbers, thus giving repeatable results.  Setting the seed to 0 or blank provides a random starting seed. 

Releasing resources

Be careful when you release resources.  The Resource Pool Release block is used to return the resource back to the appropriate pool when the resource is no longer required.  If the block immediately downstream from the Resource Pool Release block has a capacity limit and is full, the resource won’t get released when it is finished with the upstream task.  The resource won’t get released until the item with the resource can move past the Resource Pool Release block and into the next residence block downstream.  This issue is easy to overlook.  It will prevent the resource from being used elsewhere even though it is done with its current task.  The Resource Pool will overly estimate its utilization.  I suggest you specifically look for this issue at the end of model development on every model.  The following construct is one way to eliminate the issue.  In this construct, when the Activity for Task X is busy, the item will be in the Activity labeled "Task X Busy".  When the Activity for Task X is blocked, it will wait in the queue labeled "Task X Blocked" while the Gate prevents anything new from arriving into the Activity block.

  

Debugging infinite loops where the simulation time never progresses past a certain point

Occasionally your model might have a bug which causes it to go into an infinite loop.  This loop could be due to several reasons.  The reasons could involve either creating an infinite number of items, an item looping continuously in a circle, from an unending for loop, or from several other reasons.  There is an easy way to figure out where the loop is occurring.  Turn on the profiler and run the model again well into the infinite loop.  After a while, you must manually stop the model but it should show you the block (S) where the excessive time has been spent.  The blocks with excessive times should be where the infinite loop is occurring. 

Online Resources

I taught a one-on-one ExtendSim class a few weeks ago.  The student asked about ONLINE resources a new modeler should be aware of.  Great question.  It was an interesting discussion I want to share.

One of the online resources I suggested was the Winter Simulation Conference Proceeding archives. Coincidentally, I noticed that the proceedings from the last conference (2017) has recently been posted.  There are papers on basic simulation concepts that is a must read for everyone!  Even for the advanced modeler, these concepts are a great refresher from time to time.  Here is the short list of suggestions.

Winter Simulation Conference Proceedings (https://informs-sim.org/)

• A Tutorial on How to Select Simulation Input Probability Distributions by Averill Law (2016)
• Inside Discrete-event Simulation Software: How It Works and Why It Matters by Tom Schriber (2017)
• A Tutorial on Simulation Conceptual Modeling by Stewart Robinson (2017)
• A Tutorial on the Operational Validation of Simulation Models by Robert Sargent (2016)
• A Practical Look at Simulation Project Management by Joseph Hugan (2014)
• Information and Process Modeling for Simulation by Gerd Wagner (2014)
• How to Build Valid and Credible Simulation Models by Averill M. Law (2009)

There are also many advanced topics that I think are interesting that you can find information on WSC.  I recently become interested in agent based simulation.  I attended several presentations on this topic this past year at WSC.  After attending these presentations and doing some studying up on the topic, I built an ABS model myself in ExtendSim.  I discovered ExtendSim was a great fit for a certain type of agent based models.  I used some of the advanced database features along with the Queue Equation block and the Query Equation blocks.  If anyone is interested in agent based modeling then I suggest you look up the WSC paper below and look for other good articles on the WSC website.

• Agent-Based Modeling and Simulation by Charles M. Macal and Michael J. North (2015)

I also pointed out to my student there are lots of papers on actual customer models.  We did a search together on the site and found his company had published a few papers on WSC about a decade ago.  What a coincidence.  It turned out to be a great resource for him. 

Here are other great online resources.

• ExtendSim Exchange (https://www.extendsim.com/forums/)
• ExtendSim Youtube Channel (https://www.youtube.com/user/ExtendSim/featured)
• LinkedIn ExtendSim Group (https://www.linkedin.com/groups/1705447)
• Blogspot (http://simulationsoftware.blogspot.com/
• ExtendSim Industries Page (https://www.extendsim.com/solutions/industries)

You might be reading this post on either Linked In or the Blogspot - so you might know about one and not the other - so I posted both.  And if you have other online resources that I have not mentioned then please share by adding it to the comments!

To Animate or not to Animate

Animation is an important feature of a simulation model. And it’s a great way to introduce a model to someone who has never seen a simulation before. Well-thought-out model animation can grab someone’s attention like nothing else and can help that person see how a system works. I have found that many new modelers believe the primary purpose of animation is to serve as eye candy once the model has been developed. However, animation should play a much bigger role than just being a selling point of the model.  

Animation should be a critical component throughout the model-building process, in both debugging and validation. My philosophy is that if you want to monitor something, then animate it! If it is an important result, then animate it! If something is not animated, then there is a good chance you will IGNORE it for most simulation runs! So let’s discuss some ways animation can be used to our advantage in these situations.

Item animation is very useful in debugging the routing of items. For example, if we have a system where only red items should be traveling through the first path and only blue items should be traveling through the second path, item animation can show that, indeed, the red and blue are going through the right paths.

The alternative is to constantly check out History blocks strategically placed throughout the model. History blocks are great, but you must open the History block up and inspect the results for it to be useful! How often is this actually done? I inspect the History blocks often but not all of them on every run.

Also notice the animation of the numbers on the Queue and Activity blocks above. This shows the actual number of items in each block. It is useful for finding bottlenecks in the system, but often, these numbers are hidden inside h-blocks. However, there is a way we can still monitor important statistics when the blocks are inside h-blocks.   

Below is a good example of h-block animation for the purposes of debugging. This is the Emergency Department model from the ExtendSim \ Examples \ Discrete Event folder.  Check this model out sometime to see how it has been constructed.  Notice the results that are posted on top of the h-blocks. This animation would be impossible to overlook.  

The resulting patients-per-day amount is clearly animated (shown by the red arrow) when we run the model.  Let’s say we made a change to the logic of how we segregated the Main ED patients (labeled as ED Room) and the Fast Track patients (labeled as FT Room). If there were a problem with our new logic, then that would be OBVIOUS from the animation. If we didn’t animate it, then we might not notice the bug until much later. With my luck, I wouldn’t notice it until I had forgotten about the change to the logic, and at that point, the time to debug would be significant!  Animation can help bring bugs to our attention faster.

Now let’s talk about the high-level animation that we want to be eye-catching. Just because we build the model with the animation shown above does not mean that we can’t also put a nicely animated graphic on top. I have another version of the model shown above in which I have encapsulated the flow chart in another h-block and have animated the h-block. The model shown below is the result.

From the customer’s point of view, this type of animation is often necessary for them to visualize the system.  Not everyone can visualize how a system is working by simply watching the numbers like in the first two illustrations.  So, this must be addressed unless the model is built solely for your own consumption.

Finally, when should you think about how to animate the model? That is an important question! If I had a nickel for every time I heard someone say, “Please forgive me for the messy model. I will clean it up and add the animation later when it is finished…” Arrgh! This is a bad strategy. The most valuable benefit of animation has totally been missed.  You should use the strategies mentioned above to help you debug your model as you are building it.

Because animation is so important, you should think through your animation strategy at the beginning of the model-building process. Make it intentional and not an afterthought. Add animation to help you debug as well as the top-level animation that is eye-catching. And most importantly, don’t put off the model clean-up and animation until the end of the model; do it as you are building the model so you can take advantage of the benefits yourself.