[gl3ny]

Jon,
I truly appreciate you pointing this out to me and now it all makes sense. I didn’t think of looking at the readme file.

While I appreciate what you are saying I don’t understand why MidiTest would want to calculate using the standard deviation formula. I can only guess that it is a standard used by manufacturers of a whole host of products to calculate jitter for the purpose of providing a specification. I’m sure you are aware of how clever manufacturers like to be when it comes down to hiding numbers or true values to sell their products. MidiTest quote on their website, “MidiTest is a small utility that will test the speed and functioning of a MIDI hardware device.” That being said there should be no reason they can’t show what I am going to call ‘true jitter’ since they don’t manufacture midi interfaces.

Now that I understand MidiTest is using the standard deviation formula to calculate midi jitter, in my mind it has only strengthened the validity of the Midi Loop Test for showing true jitter.

In general terms I don’t want to knock MidiTest, I still think it’s a great app. MidiTest is basically testing your computer system with the midi interface, and it is fast and has some very useful information. However I do want to knock MidiTest as a viable tool to measure jitter.

Originally Posted by qtuner

I'm hesitant to introduce a sequencer into testing because I believe sequencer timing is independent of hardware timing. Since PC software is layered, and calling standardized APIs I think we can assume hardware performance as independent from sequencer performance. I do agree that sequencer latency is part of the sum of total latency. Is this assumption correct?

Whether this assumption is correct or not, I don’t know? But what I do know is that with a Midi Loop Test, the computer system, midi interface and the host sequencer are being tested and that is closer to a real world situation than with MidiTest. The real numbers that show when data is being received is right there in the sequencer software. After all your midi sequencer makes music, MidiTest does not.

Lets get one thing clear, I am not out to say anyone is right or wrong. To put it simply I believe I am experiencing a midi jitter problem and I’m on a mission to correct it, and that is all! Some people might think that I don’t really have a problem and that is fine everyone is entitled to an opinion. I am in contact with Steinberg and Motu regarding this problem and from the data I have provided to both parties, they agree that this should not be happing.

What matters to me is the data! People can make up their own mind if they have a problem or not.

Here are the MidiTest results ran on a Motu Midi Timepiece AV

================ Results Per Message =====================================

MESSAGES Snd Rcv Snd+Rcv

Message TotalTime: 2067.96 ms 208853.41 ms 210921.38 ms
Message MaximumTime: 0.11 ms 9.69 ms 9.72 ms
Message MinimumTime: 0.02 ms 4.77 ms 4.84 ms
Message AverageTime: 0.07 ms 6.68 ms 6.75 ms
SysexTime: 0.00 ms 0.00 ms 0.00 ms
SysexAverage: 0.00 ms 0.00 ms 0.00 ms

< 1 ms: 31250 0 0
1 - 2 ms: 0 0 0
2 - 3 ms: 0 0 0
3 - 4 ms: 0 0 0
4 - 5 ms: 0 14 10
5 - 10 ms: 0 31236 31240
10 - 20 ms: 0 0 0
20 - 50 ms: 0 0 0
50 - 100 ms: 0 0 0
> 100 ms: 0 0 0

Message count: 31250 Sysex count: 0
Sysex size: 0 Sysex passed: 0

Message latency: 6.75 ms Total time: 268.249 sec
Message jitter: 0.75 ms
Message max deviation: 2.97 ms


This is how I would calculate the data presented in MidiTest based on the same calculation used in a Midi Loop Test:

Message MaximumTime: 9.72 ms
Message MinimumTime: 4.84 ms

9.72 - 4.84 = 4.88 Message Jitter or ‘true jitter’. No fancy calculation here.



Midi Loop Test I performed consisted of 10 quantized midi note events with a length of 1/32, one on each beat lasting for 2.5 bars (1.1.1.0 to 3.2.1.0). Midi events output on MTPAV port 8 and recorded in MTPAV port 8. This was repeated 10 times giving a total of 100 note on/off events.

Here are the results of a Midi Loop Test performed immediately after the above MidiTest results.

Test 1
Message MaximumTime: 13.0 ms
Message MinimumTime: 9.0 ms
True Jitter = 4.0 ms
Test 2
Message MaximumTime: 13.0 ms
Message MinimumTime: 9.0 ms
True Jitter = 4.0 ms
Test 3
Message MaximumTime: 12.0 ms
Message MinimumTime: 7.0 ms
True Jitter = 5.0 ms
Test 4
Message MaximumTime: 12.0 ms
Message MinimumTime: 10.0 ms
True Jitter = 2.0 ms
Test 5
Message MaximumTime: 14.0 ms
Message MinimumTime: 9.0 ms
True Jitter = 5.0 ms
Test 6
Message MaximumTime: 11.0 ms
Message MinimumTime: 8.0 ms
True Jitter = 3.0 ms
Test 7
Message MaximumTime: 12.0 ms
Message MinimumTime: 8.0 ms
True Jitter = 4.0 ms
Test 8
Message MaximumTime: 12.0 ms
Message MinimumTime: 10.0 ms
True Jitter = 2.0 ms
Test 9
Message MaximumTime: 11.0 ms
Message MinimumTime: 7.0 ms
True Jitter = 4.0 ms
Test 10
Message MaximumTime: 11.0 ms
Message MinimumTime: 9.0 ms
True Jitter = 2.0 ms




If I run an average of each true jitter value that would be 3.5 ms. I have actually been generous by showing one of the tests with better results. But if make an average of the 19 total test averages just like the test above then it comes out to 3.99 ms. Wait a minute that is better than MidiTest. Could it be MidiTest itself is introducing a slight amount of true jitter, more than my sequencer?

My point is, it doesn’t matter how you want to manipulate the numbers using standard deviation or what ever formula you want to use. The bottom line is my midi notes are being recorded in my sequencer randomly within a 5ms window on top of any latency.

Originally Posted by jeamsler

Jitter should not be interpreted as the difference between the max and min deviation from norm. Those would be extreme values and represent the worst case.

I strongly disagree and believe that jitter should be interpreted by taking the max and min deviation from the norm and leave your representation of jitter should be left as a calculation for the manufactures. Those extreme values I’m afraid are a reality and there is no getting away from that.

This random 5ms window is not the full story either. The 5ms windows is a combination of the input and output jitter so each stage we could say is 2.5ms so lets look at this in a practical situation. Let’s say I want to capture a keyboard performance via midi and at a later stage play back the midi part and record it as audio.

Midi Recording - Keyboard performance > sequencer
2.5ms produced by Keyboard midi out
2.5ms produced by Midi interface input and sequencer.

Midi Playback
2.5ms produced by Sequencer + midi interface output
2.5ms produced by keyboard midi input

Keyboard generates sound and result is recorded as audio with a 10ms jitter. I think it’s widely accepted that the performance of a competent musician can be lost with anything over 5ms therefore the jitter is unacceptable to me.

My RayDat on the other hand would produce 4ms using the same calculations which would be acceptable. However keyboard jitter is not the issue here but could be measured after establishing a base line with your midi interface.

My conclusion is that MidiTest is most definitely not a valid tool for testing midi jitter. I advise anyone who is looking to purchase a midi interface and who is concerned with jitter not buy one on the strength of the results produced by MidiTest.

And for anyone who is looking at latency alone when choosing a midi interface. Just remember this, jitter is random latency!