Sysex dump issue

[Measuring Man]

We're in the process of setting up our mastering gear to dump sysex midi dumps to our software for backup purposes. We have a TC6000, Weiss DS1, Weiss EQ1, and a Digital Domain DD-2.

Here's my question: How do you know if the sysex dump into the computer went error free? The DS1 and EQ1 have error checking for when loading into memory. However I was thinking, what if the dump into the computer went wrong. You wouldn't know until trying to load it back in, at which point everything would be FUBAR.

Anyone have any experience trying this? Lessons, tips etc?

[danika]

Do it twice, calculate the CRC for each file, and see if they match.

[bob katz]

Originally Posted by danika

Do it twice, calculate the CRC for each file, and see if they match.

The CRC is calculated automatically in the sequencer software. It could get real time-consuming to compare it and probably not worth the trouble. I usually take my chances and trust things, using a Sequencer and system that has proved to be reliable (up to now!!!!). Worst case scenario, you're working a whole day manually figuring out settings. Yes, that's a pain, but it hasn't happened to me due to a technical glitch or corruption in a long time. But it HAS happened to me due to human error more than once. It'll get hectic around here, and I (or an assistant, god forbid) may forget a procedure. Builds a lot of character then! No amount of "crc check" is going to fix a human error or oversight. You're asking to add about an hour of extra work to your day, and in that case you'd be better off writing down all your processor settings with an electric pencil!

BK

[Measuring Man]

A CRC in the sequencer was what I was wondering about. I know a fair bit about MIDI, but I've never dealt with Sysex too much before. Doing it manually is waaaay to much.

So to clarify, let me ask...do sequencers have any kind of Checksum for sysex? And if not, Bob, you're saying you've been doing it for years and the only mess up has been human? Encouraging, well it would be if I didn't mess things up so often.

Anyone automating parameters using MIDI? If you had to jump a threshold up for a solo, do these mastering units respond in time and consistently?

[bob katz]

Originally Posted by Measuring Man

A CRC in the sequencer was what I was wondering about. I know a fair bit about MIDI, but I've never dealt with Sysex too much before. Doing it manually is waaaay to much.

So to clarify, let me ask...do sequencers have any kind of Checksum for sysex? And if not, Bob, you're saying you've been doing it for years and the only mess up has been human? Encouraging, well it would be if I didn't mess things up so often.

Anyone automating parameters using MIDI? If you had to jump a threshold up for a solo, do these mastering units respond in time and consistently?

You're talking to nearly the only guy on the planet who does know how to and can automate parameters using MIDI. There is a small but vocal group of mastering engineers who also recognize the power of this ability, but the majority of mastering engineers scoff at it and continue on their merry ways.

Anyway, my answer is yes. It is worth it, if you can pass the learning barrier. Regardless, ALWAYS CAPTURE THE RESULT OF YOUR MASTERING TO A FILE, NEVER RELY ON A COMPUTER TO KEEP ANY PARAMETERS, plugin settings, automation or routing over a long period of time.

[Measuring Man]

Do you get any lag you have to compensate for in MIDI. Anticipating automation moves so that they happen on time?

On a side note, similar to your surprise that nobody does it more, we here find it so time consuming and awkward to have a separate playback system from recording. You get used to instant lock, and having your masters with the unmastered track so you can reference and just generally move around faster. But I digress....

[Ben B]

Originally Posted by bob katz

You're talking to nearly the only guy on the planet who does know how to and can automate parameters using MIDI.

I'm always surprised at how many engineers dismiss the usefulness of MIDI. I've been using sysex to automate outboard devices for as long as I can remember.

BTW - have you ever checked out the sysex faders/buttons, etc, in the Logic Environment window? You can build complete editors, fully automatable, for just about anything you can think of. It's kind of like the StudioWare panels in Sonar.

I'm primarily a PT user, but these features in Logic (and Sonar) are awesome. In PT, sysex is essentially limited to straight-up recording and playback. Fine for bulk dumps, or for recording automation moves from outboard processors. Not quite as cool as being able to set up editors and controllers right from within the software though.

-Ben B

[kosmokrator]

Originally Posted by Measuring Man

We're in the process of setting up our mastering gear to dump sysex midi dumps to our software for backup purposes. We have a TC6000, Weiss DS1, Weiss EQ1, and a Digital Domain DD-2.

Here's my question: How do you know if the sysex dump into the computer went error free? The DS1 and EQ1 have error checking for when loading into memory. However I was thinking, what if the dump into the computer went wrong. You wouldn't know until trying to load it back in, at which point everything would be FUBAR.

Anyone have any experience trying this? Lessons, tips etc?

I do that daily with a SSL AWS900 (Total recall + automation), A 6000 and a Eventide H8000.
I've NEVER had the sequencer (PT) screw up the sysex.
But, as Bob said, sometimes the user is the problem...

[synthoid]

A special-purpose sysex capture program for this purpose is worth it IMO. I use this same capability for capturing synth settings and I have a ton of them.

On a Mac, I recommend SysEx Librarian. It's free! It gives you message counts and other stuff that will generally give you peace of mind. Most often the sysex dump from a particular piece of gear will have a fixed size or a few fixed sizes, so once you've got checksum verification and the right size, you can be pretty confident you've got the thing properly captured.

On a PC, the standard is MIDIOX. I've never used it but it has a good reputation.

-synthoid

[Measuring Man]

Originally Posted by synthoid

A special-purpose sysex capture program for this purpose is worth it IMO. I use this same capability for capturing synth settings and I have a ton of them.

On a Mac, I recommend SysEx Librarian. It's free! It gives you message counts and other stuff that will generally give you peace of mind. Most often the sysex dump from a particular piece of gear will have a fixed size or a few fixed sizes, so once you've got checksum verification and the right size, you can be pretty confident you've got the thing properly captured.

On a PC, the standard is MIDIOX. I've never used it but it has a good reputation.

-synthoid

Excellent suggestion, I'll have to look into that. This has been the kind of advice and help I was hoping for. Thanks everyone who suggested stuff, it's all great stuff.

[Ben B]

Just curious -- has anyone actually experienced significant errors doing bulk dumps? I'd be kind of surprised to hear this. Relatively speaking, it's just not that much data.

-Ben B

[wado1942]

You know, I was just having a conversation with another engineer friend of mine about MIDI. I've never in my life used MIDI (in session anyway) for anything other than hooking 2 keyboards together with similar patches and different signal paths to thicken the sound. I borrow a synth once and a while that lets you program patches in a computer and dump them to the synth through MIDI. I've also used an SMPTE->MTC box to practice slaving my 16-track to computer but have yet to use it in session.

This other engineer, on the other hand, uses MIDI for all sorts of logic functions that I can't even begin to explain. I admit, aside from my computer, my sync box and one of my reverbs, I don't have any actual MIDI capable equipment in my studio. Guess I never saw the need for it. Where does one even learn about programming and automating hardware perameters with MIDI?

[synthoid]

Originally Posted by Ben B

Just curious -- has anyone actually experienced significant errors doing bulk dumps? I'd be kind of surprised to hear this. Relatively speaking, it's just not that much data.

I have experienced lots of problems capturing sysex data with sequencers. It was a notorious problem with hardware sequencers, which had inadequate buffering to capture big sysex dumps. It got a little better with software sequencers, but the still many (especiallly older) MIDI interfaces were bad about buffering sysex data that was coming in too fast.

The reverse problem happens sometimes too: the gear that generates the sysex dump can sometimes be unable to absorb it as fast as a sequencer wants to transmit it, when restoring.

-synthoid

[Ben B]

Fascinating. I've used sysex functions in Pro Tools, Vision, Logic, Cakewalk/Sonar, DP, etc, and have never once experienced this kind of problem. Just lucky, I guess.

-Ben B

[Ben B]

Just to add to my above post, I never tried this with a hardware sequencer. Back in the day, I had a Yamaha MIDI data filer (I forget the model number) that I used for bulk dumps, which could then be saved to floppies. I've only used software sequencers for realtime sysex parameter control, and have never had a problem (except for a known problem in a certain version of Pro Tools that has been corrected). I wonder if it's really the interfaces that are to blame, in general.

-Ben B

[synthoid]

Originally Posted by Ben B

Just to add to my above post, I never tried this with a hardware sequencer. Back in the day, I had a Yamaha MIDI data filer (I forget the model number) that I used for bulk dumps, which could then be saved to floppies. I've only used software sequencers for realtime sysex parameter control, and have never had a problem (except for a known problem in a certain version of Pro Tools that has been corrected). I wonder if it's really the interfaces that are to blame, in general.

-Ben B

Things seem to be a lot better with software sequencers than they were with hardware. I had a QY700 and an MPC4000 and neither of them was capable of recording a sysex dump from an Access Virus (those dumps are quite large). I had some problems with Logic 7 and an older MIDI interface, but these days I don't notice any problems.

That said, I still use SysEx Librarian because this is an archival function and it's really important to get it right.

MIDI is wonderful in some ways but it has been left in the dust by audio, in terms of performance and robustness. That's just plain weird if you ask me, haha. One channel of 96KHz audio in digital format has enough bandwidth for hundreds of MIDI channels running with perfect timing accuracy. With MIDI interfaces we get crappy jitter figures and stomach-aches about sysex dumps, haha.

-synthoid