Reviewing the Review

If you are evaluating simulation software, a third-party review or paper comparing the features of different simulation software programs can be very useful. However, not all software reviews are created equal. Some are truly excellent, well researched, informative, and accurate. Others contain factual errors, use outdated software, and are even sometimes biased by economic considerations of the author.

The gold standard for a comparison is Tom Schriber and Dan Brunner’s perennial “Inside Simulation Software: How it Works and Why it Matters”. This paper does not attempt to recommend one software product over another, but it does give insight into the inner workings of a variety of simulation packages. When Tom and Dan were including ExtendSim (Extend) into their paper, we exchanged nearly 100 emails detailing the precise behavior of our software. The authors have taken care to update the paper as new versions of ExtendSim have come out. There are other good reviews as well. I have not always agreed with their conclusions, but I do respect their methodology.

I won’t reference the paper on the other end of the spectrum. However, I did find 14 obvious technical errors in a little more than a page of text describing ExtendSim. These included the wrong web site, incorrectly stating that there were limitations on the number of levels of hierarchy, and even the product name was incorrect. The author never contacted us or asked us to comment on his review.

So, if you are looking at simulation software reviews, take the following steps to make sure that you are getting a thoughtful, accurate commentary:

• Contact the author and ask them if the paper was reviewed by the simulation software vendors before publication.
• Look for obvious technical errors. Generally, this is an indication that the author did not do their homework.
• Did they use the latest version of the software?
• Does the author have any connection with a simulation software vendor, currently or in the past?
• If you can, contact modelers who use the software in the paper. Ask if they agree with the conclusions.

As software developers, you can expect us to be biased towards our own creations. Unfortunately, you cannot always find that even in neutral third party evaluations. Caveat emptor.

Bias block’s use in optimization problems

I feel the bias block does not get the use it deserves and would like to make a case for it.

If you do not use the Rate library, there may be no point trying to understand the Bias block…However, lack of familiarity rather than true functionality might be the reason you’re not using Rate. The ExtendSim manual has a good discussion of ExtendSim AT’s discrete rate capabilities; you may want to read up on it a bit. For now let focus on this little block called the Bias block.

The block is small and simple but carries a powerful concept. The bias block allows the rate of flow through particular model sections to be maximized. In other words, after defining where the flow should “preferably” go, the model optimizes the direction of the flow.

To illustrate my point lets give a practical example:

-Goal: The model purpose is to properly allocate power supply so that each customer gets what they need at the lowest possible cost.

-Setting: 3 different sources of power, 4 pools of users and a distribution network.

-Logic: Each source of power has an associated cost and is linked to the distribution network. Each pool of users needs a certain amount of power and is connected to the distribution network.

Supply information	Maximum Megawatt-hour	Cost per Megawatt	Bias priority
Supply 1	50 MW-h	35 $/MW-h	1
Supply 2	50 MW-h	50 $/MW-h	2
Supply 3	50 MW-h	60 $/MW-h	3

Demand information	Megawatt-hour Demand
User a	20 MW-h
User b	30 MW-h
User c	20 MW-h
User d	10 MW-h

-Implementation: The modeler will construct the distribution system, the suppliers and the consumers… When all of that is done, the modeler can position a bias block next to each power supply; the lowest cost producer will be given the highest priority and so on…. And that’s it; the model is done and will distribute energy at the lowest cost.

The concept of bias block allows a modeler to look at his model as a global system. What is flowing through the system will follow the preferences provided by the bias block.

If the Bias block gets used more, I have in mind some fun upgrades to make the block even more powerful. Right now, the Bias parameter only authorized priorities between different parts of the model. It would be a fun new development to allow a full “objective function” to be defined which would be either maximized or minimized.

Simulation and Innovation

I remember, very clearly, the first moment that I saw Extend (now, of course, called ExtendSim). It was at a TIMS (a precursor of INFORMS) conference in 1993. At that moment I realized that I was looking at the future of simulation. This was an amazing program, a quantum leap in simulation technology. You could build a model by dragging and connecting blocks together. You could combine blocks together to create a single block. You could interactively change the model while it was running. You could create your own blocks. You could even program a block to do something when the simulation was not running. Wow!

At that point, I had already worked with a number of simulation programs (Siman, Slam, GPSS and more) but none of them could compare to the capabilities of Extend. What did I do? I asked for a job. After a rigorous interview process (I met everyone at Imagine that and did a little programming) and some pretty tough negotiations (they offered, I accepted), I started work. That was 17 years ago. The most amazing part of this story is that we’re still ahead of the pack. I’m even seeing some new simulation programs that look eerily like Extend did in 1993. I’ve heard some say that there has been no significant innovation in simulation software in the last 25 years. Extend came out in the 1980's - perhaps they are right.

Sense-itivity Analysis

Today I read a headline that said “75% Probability Apple Stays With AT&T”. That was based on: “"couldn’t find compelling evidence" that AT&T's contract with Apple ends this year. He gives it a 50% chance. Additionally, there's a 25% chance that AT&T would bid for -- and win -- another year of exclusivity. Add them up, you get 75%”.

Whenever somebody adds up probabilities, that sets off an alarm in my head. What if the analyst believed that there was a 60% probability that AT&T would get the new contract? Then the sum of the probabilities would be 110%. That’s a little more than absolute certainty, which never exists in the stock market.

The lesson here is to use a little, what I will call, “sense-itivity analysis”. Plug some plausible numbers in to make sure that the answer still passes a common sense test. We know that the probability must be 100% or less. In this case, it will certainly be less than 100%. So the result of 110% tells us that there is a math problem here. Doing a little “sense-itivity analysis” is a good idea in simulation models as well as the stock market.

For the record, the correct answer is 62.5%. I will let you work out the math.