MIDI Time Code Spec MIDI Time Code and Cueing Detailed Specification (Supplement to MIDI 1.0) 12 February 1987 Justification For MIDI Time Code and Cueing The merit of implementing the MIDI Time Code proposal within the current MIDI specification is as follows: SMPTE has become the de facto timing reference standard in the professional audio world and in almost the entire video world. SMPTE is also seeing more and more use in the semi-professional audio area. We hope to combine this universal timing reference, SMPTE, with the de facto standard for controlling musical equipment, MIDI. Encoding SMPTE over MIDI allows a person to work with one timing reference throughout the entire system. For example, studio engineers are more familiar with the idea of telling a multitrack recorder to punch in and out of record mode at specific SMPTE times, as opposed to a specific beat in a specific bar. To force a musician or studio engineer to convert back and forth between a SMPTE time and a specific bar number is tedious and should not be necessary (one would have to take into account tempo and tempo changes, etc.). Also, some operations are referenced only as SMPTE times, as opposed to beats in a bar. For example, creating audio and sound effects for video requires that certain sounds and sequences be played at specific SMPTE times. There is no other easy way to do this with Song Position Pointers, etc., and even if there was, it would be an unnatural way for a video or recording engineer to work. MIDI Time Code is an absolute timing reference, whereas MIDI Clock and Song Position Pointer are relative timing references. In virtually all audio for film/video work, SMPTE is already being used as the main time base, and any musical passages which need to be recorded are usually done by getting a MIDI-based sequencer to start at a pre-determined SMPTE time code. In most cases, though, it is SMPTE which is the Master timing reference being used. In order for MIDI-based devices to operate on an absolute time code which is independent of tempo, MIDI Time Code must be used. Existing devices merely translate SMPTE into MIDI Clocks and Song Position Pointers based upon a given tempo. This is not absolute time, but relative time, and all of the SMPTE cue points will change if the tempo changes. The majority of sound effects work for film and video does not involve musical passages with tempos, rather it involves individual sound effect "events" which must occur at specific, absolute times, not relative to any "tempo". MIDI Time Code System Components SMPTE to MTC Converter This box would either convert longitudinal (audio-type) or vertical (video-type) SMPTE time code from a master timing device into MTC. The function could be integrated into video tape recorders (VTRs) or syncronization units that control audio tape recorders (ATRs). Alternately, a stand-alone box would do the conversion, or simply generate MTC directly. Note that conversion from MTC to SMPTE time code is not envisioned, as it is of little practical value. Cue List Manager This would be a device or computer program that would maintain a cue list of desired events, and send the list to the slaves. For performance, the manager might pass the Time Code from the SMPTE-MTC converter through to the slaves, or, in a stand-alone system it might generate Time Code itself. This "central controller" would presumably also contain all library functions for downloading sound programs, samples, sequences, patterns, and so on, to the slaves. A Cue List Manager would pre-load intelligent MTC peripherals (see below) with this data. MTC Sequencer To control existing equipment or any device which does not recognize MTC in an MTC system, this device would be needed. It would receive the cue list from the manager, and convert the cues into normal MIDI commands. At the specified SMPTE times, the sequencer would then send the MIDI commands to the specific devices. For example, for existing MIDI equipment it might provide MIDI messages such as Note On, Note Off, Song Select, Start, Stop, Program Changes, etc. Non-MIDI equipment (such as CD players, mixing consoles, lighting, sound effects cartridge units and ATRs) may also be controlled if such a device had relay controls. Intelligent MTC Peripheral In this category belong devices capable of receiving an MTC Cue List from the manager, and triggering themselves appropriately when the correct Time Code (SMPTE or MIDI) has been received. Above this minimum, the device might be able to change its programming in response to the Cue List, or prepare itself for ensuing events. For example, an intelligent MTC-equipped analog multitrack tape machine might read in a list of punch in/punch out cues from the Cue List Manager, and then alter then to internally compensate for its bias current rise and fall times. A sampling-based sound effects device might preload samples from its own disk drive into a RAM buffer, in anticipation of needing them for cues later on in the cue list. It should be mentioned that while these functions are separately described, actual devices may incorporate a mixture of these functions, suited to specific applications in their market. A MIDI Time Code System The MIDI Time Code format contains two parts: Time Code and Set Up. Time Code is relatively straightforward: hours, minutes, seconds and frame numbers (approximately 1/30 of a second) are encoded and distributed throughout the MIDI system so that all the units know exactly what time it is. Set Up, however, is where MTC gains its power. It is a format for informing MIDI devices of events to be performed at specific times. Ultimately, this aspect of MTC will lead to the creation of an entirely new class of production equipment. Before getting into the nuts and bolts of the spec itself, let's talk about some of the uses and features of forthcoming devices that have been envisioned. Set Up begins with the concept of a cue list. In video editing, for example, it is customary to transfer the video master source tapes, which may be on expensive, two-inch recorders, to less-expensive recorders. The editing team then works over this copy, making a list of all the segments that they want to piece together as they are defined by their SMPTE times. For example, the first scene starts at time A and ends at time B, the next scene starts at time C and ends at time D. A third scene may even lie between the first two. When done, they feed this cue list time information into the editing system of the master recorder(s) or just give the cue list to an editor who does the work manually. The editing system or editor then locates the desired segments and assembles them in the proper sequence. Now suppose that instead of one or two video recorders, we have twenty devices that will play a part in our audio/video or film production: special effects generators for fades and superimpositions, additional decks with background scenery, live cameras, MIDI sequencers, drum machines, synthesizers, samplers, DDLs, soundtrack decks, CDs, effects devices, and so on. As it stands now, each of these devices must be handled more or less separately, with painstaking and time-consuming assembly editing or multitrack overdubs. And when a change in the program occurs (which always happens), anywhere from just a few items to the whole system may need to be reprogrammed by hand. This is where MIDI Time Code comes in. It can potentially control all of these individual production elements so that they function together from a single cue list. The master controller which would handle this function is described as a Cue List Manager. On such a console, you would list what you want each device to do, and when to do it. The manager would then send the cue list to the various machines via the MTC Set Up protocol. Each unit would then react as programmed when the designated MIDI Time Code (or conventional SMPTE Time Code) appears. Changes? No problem. Simply edit the cue list using simple word-processing functions, then run the tape again. MTC thus integrates into a manageable system all of the diverse tools at our disposal. It would drastically reduce the time, money and frustration needed to produce a film or video. Having covered the basic aspects of a MIDI Time Code system, as well as examples of how an overall system might function, we will now take a look at the actual MIDI specification itself. MIDI Time Code For device synchronization, MIDI Time Code uses two basic types of messages, described as Quarter Frame and Full. There is also a third, optional message for encoding SMPTE user bits. Quarter Frame Messages Quarter Frame messages are used only while the system is running. They are rather like the PPQN or MIDI clocks to which we are accustomed. But there are several important ways in which Quarter Frame messages differ from the other systems. As their name implies, they have fine resolution. If we assume 30 frames per second, there will be 120 Quarter Frame messages per second. This corresponds to a maximum latency of 8.3 milliseconds (at 30 frames per second), with accuracy greater than this possible within the specific device (which may interpolate inbetween quarter frames to "bit" resolution). Quarter Frame messages serve a dual purpose: besides providing the basic timing pulse for the system, each message contains a unique nibble (four bits) defining a digit of a specific field of the current SMPTE time. Quarter frames messages should be thought of as groups of eight messages. One of these groups encodes the SMPTE time in hours, minutes, seconds, and frames. Since it takes eight quarter frames for a complete time code message, the complete SMPTE time is updated every two frames. Each quarter frame message contains two bytes. The first byte is F1, the Quarter Frame System Common byte. The second byte contains a nibble that represents the message number (0 through 7), and a nibble for one of the digits of a time field (hours, minutes, seconds or frames). Quarter Frame Messages (2 bytes): F1 F1 = Currently unused and undefined System Common status byte = 0nnn dddd dddd = 4 bits of binary data for this Message Type nnn = Message Type: 0 = Frame count LS nibble 1 = Frame count MS nibble 2 = Seconds count LS nibble 3 = Seconds count MS nibble 4 = Minutes count LS nibble 5 = Minutes count MS nibble 6 = Hours count LS nibble 7 = Hours count MS nibble and SMPTE Type After both the MS nibble and the LS nibble of the above counts are assembled, their bit fields are assigned as follows: FRAME COUNT: xxx yyyyy xxx = undefined and reserved for future use. Transmitter must set these bits to 0 and receiver should ignore! yyyyy = Frame number (0-29) SECONDS COUNT: xx yyyyyy xx = undefined and reserved for future use. Transmitter must set these bits to 0 and receiver should ignore! yyyyyy = Seconds Count (0-59) MINUTES COUNT: xx yyyyyy xx = undefined and reserved for future use. Transmitter must set these bits to 0 and receiver should ignore! yyyyyy = Minutes Count (0-59) HOURS COUNT: x yy zzzzz x = undefined and reserved for future use. Transmitter must set this bit to 0 and receiver should ignore! yy = Time Code Type: 0 = 24 Frames/Second 1 = 25 Frames/Second 2 = 30 Frames/Second (Drop-Frame) 3 = 30 Frames/Second (Non-Drop) zzzzz = Hours Count (0-23) Quarter Frame Message Implementation When time code is running in the forward direction, the device producing the MIDI Time Code will send Quarter Frame messages at quarter frame intervals in the following order: F1 0X F1 1X F1 2X F1 3X F1 4X F1 5X F1 6X F1 7X after which the sequence repeats itself, at a rate of one complete 8-message sequence every 2 frames (8 quarter frames). When time code is running in reverse, the quarter frame messages are sent in reverse order, starting with F1 7X and ending with F1 0X. Again, at least 8 quarter frame messages must be sent. The arrival of the F1 0X and F1 4X messages always denote frame boundaries. Since 8 quarter frame messages are required to definitely establish the actual SMPTE time, timing lock cannot be achieved until the reader has read a full sequence of 8 messages, from first message to last. This will take from 2 to 4 frames to do, depending on when the reader comes on line. During fast forward, rewind or shuttle modes, the time code generator should stop sending quarter frame messages, and just send a Full Message once the final destination has been reached. The generator can then pause for any devices to shuttle to that point, and resume by sending quarter frame messages when play mode is resumed. Time is considered to be "running" upon receipt of the first quarter frame message after a Full Message. Do not send quarter frame messages continuously in a shuttle mode at high speed, since this unnecessarily clogs the MIDI data lines. If you must periodically update a device's time code during a long shuttle, then send a Full Message every so often. The quarter frame message F1 0X (Frame Count LS nibble) must be sent on a frame boundary. The frame number indicated by the frame count is the number of the frame which starts on that boundary. This follows the same convention as normal SMPTE longitudinal time code, where bit 00 of the 80-bit message arrives at the precise time that the frame it represents is actually starting. The SMPTE time will be incremented by 2 frames for each 8-message sequence, since an 8-message sequence will take 2 frames to send. Another way to look at it is: When the last quarter frame message (F1 7X) arrives and the time can be fully assembled, the information is now actually 2 frames old. A receiver of this time must keep an internal offset of +2 frames for displaying. This may seem unusual, but it is the way normal SMPTE is received and also makes backing up (running time code backwards) less confusing - when receiving the 8 quarter frame messages backwards, the F1 0X message still falls on the boundary of the frame it represents. Each quarter frame message number (0->7) indicates which of the 8 quarter frames of the 2-frame sequence we are on. For example, message 0 (F1 0X) indicates quarter frame 0 of frame #1 in the sequence, and message 4 (F1 4X) indicates quarter frame 1 of frame #2 in the sequence. If a reader receives these message numbers in descending sequence, then it knows that time code is being sent in the reverse direction. Also, a reader can come on line at any time and know exactly where it is in relation to the 2-frame sequence, down to a quarter frame accuracy. It is the responsibility of the time code reader to insure that MTC is being properly interpreted. This requires waiting a sufficient amount of time in order to achieve time code lock, and maintaining that lock until synchronization is dropped. Although each passing quarter frame message could be interpreted as a relative quarter frame count, the time code reader should always verify the actual complete time code after every 8-message sequence (2 frames) in order to guarantee a proper lock. For example, let's assume the time is 01:37:52:16 (30 frames per second, non-drop). Since the time is sent from least to most significant digit, the first two Quarter Frame messages will contain the data 16 (frames), the second two will contain the data 52 (seconds), the third two will represent 37 (minutes), and the final two encode the 1 (hours and SMPTE Type). The Quarter Frame Messages description defines how the binary data for each time field is spread across two nibbles. This scheme (as opposed to simple BCD) leaves some extra bits for encoding the SMPTE type (and for future use). Now, let's convert our example time of 01:37:52:16 into Quarter Frame format, putting in the correct hexadecimal conversions: F1 00 F1 11 10H = 16 decimal F1 24 F1 33 34H = 52 decimal F1 45 F1 52 25H = 37 decimal F1 61 F1 76 01H = 01 decimal (SMPTE Type is 30 frames/non-drop) (note: the value transmitted is "6" because the SMPTE Type (11 binary) is encoded in bits 5 and 6) For SMPTE Types of 24, 30 drop frame, and 30 non-drop frame, the frame number will always be even. For SMPTE Type of 25, the frame number may be even or odd, depending on which frame number the 8-message sequence had started. In this case, you can see where the MIDI Time Code frame number would alternate between even and odd every second. MIDI Time Code will take a very small percentage of the MIDI bandwidth. The fastest SMPTE time rate is 30 frames per second. The specification is to send 4 messages per frame - in other words, a 2-byte message (640 microseconds) every 8.333 milliseconds. This takes 7.68 % of the MIDI bandwidth - a reasonably small amount. Also, in the typical MIDI Time Code systems we have imagined, it would be rare that normal MIDI and MIDI Time Code would share the same MIDI bus at the same time. Full Message Quarter Frame messages handle the basic running work of the system. But they are not suitable for use when equipment needs to be fast-forwarded or rewound, located or cued to a specific time, as sending them continuously at accelerated speeds would unnecessarily clog up or outrun the MIDI data lines. For these cases, Full Messages are used, which encode the complete time into a single message. After sending a Full Message, the time code generator can pause for any mechanical devices to shuttle (or "autolocate") to that point, and then resume running by sending quarter frame messages. Full Message - (10 bytes) F0 7F 01 hr mn sc fr F7 F0 7F = Real Time Universal System Exclusive Header = 7F (message intended for entire system) 01 = , 'MIDI Time Code' = 01, Full Time Code Message hr = hours and type: 0 yy zzzzz yy = type: 00 = 24 Frames/Second 01 = 25 Frames/Second 10 = 30 Frames/Second (drop frame) 11 = 30 Frames/Second (non-drop frame) zzzzz = Hours (00->23) mn = Minutes (00->59) sc = Seconds (00->59) fr = Frames (00->29) F7 = EOX Time is considered to be "running" upon receipt of the first Quarter Frame message after a Full Message. User Bits "User Bits" are 32 bits provided by SMPTE for special functions which vary with the application, and which can be programmed only from equipment especially designed for this purpose. Up to four characters or eight digits can be written. Examples of use are adding a date code or reel number to the tape. The User Bits tend not to change throughout a run of time code. User Bits Message - (15 bytes) F0 7F 01 u1 u2 u3 u4 u5 u6 u7 u8 u9 F7 F0 7F = Real Time Universal System Exclusive Header = 7F (message intended for entire system) 01 = , MIDI TIme Code = 02, User Bits Message u1 = 0000aaaa u2 = 0000bbbb u3 = 0000cccc u4 = 0000dddd u5 = 0000eeee u6 = 0000ffff u7 = 0000gggg u8 = 0000hhhh u9 = 000000ii F7 = EOX These nibble fields decode in an 8-bit format: aaaabbbb ccccdddd eeeeffff gggghhhh ii. It forms 4 8-bit characters, and a 2 bit Format Code. u1 through u8 correspond to SMPTE Binary Groups 1 through 8. u9 are the two Binary Group Flag Bits, as defined by SMPTE. This message can be sent whenever the User Bits values must be transferred to any devices down the line. Note that the User Bits Message may be sent by the MIDI Time Code Converter at any time. It is not sensitive to any mode. MIDI Cueing MIDI Cueing uses Set-Up Messages to address individual units in a system. (A "unit" can be be a multitrack tape deck, a VTR, a special effects generator, MIDI sequencer, etc.) Of 128 possible event types, 19 are currently defined. Set-Up Messages (13 bytes plus any additional information): F0 7E 04 hr mn sc fr ff sl sm F7 F0 7E = Non-Real Time Universal System Exclusive Header = Channel number 04 = , MIDI Time Code = Set-Up Type 00 = Special 01 = Punch In points 02 = Punch Out points 03 = Delete Punch In point 04 = Delete Punch Out point 05 = Event Start points 06 = Event Stop points 07 = Event Start points with additional info. 08 = Event Stop points with additional info. 09 = Delete Event Start point 0A = Delete Event Stop point 0B = Cue points 0C = Cue points with additional info 0D = Delete Cue point 0E = Event Name in additional info hr = hours and type: 0 yy zzzzz yy = type: 00 = 24 Frames/Second 01 = 25 Frames/Second 10 = 30 Frames/Second drop frame 11 = 30 Frames/Second non-drop frame zzzzz = Hours (00-23) mn = Minutes (00-59) sc = Seconds (00-59) fr = Frames (00-29) ff = Fractional Frames (00-99) sl, sm = Event Number (LSB first) F7 = EOX Description of Set-Up Types: 00 Special refers to the set-up information that affects a unit globally (as opposed to individual tracks, sounds, programs, sequences, etc.). In this case, the Special Type takes the place of the Event Number. Five are defined. Note that types 01 00 through 04 00 ignore the event time field. 00 00 Time Code Offset refers to a relative Time Code offset for each unit. For example, a piece of video and a piece of music that are supposed to go together may be created at different times, and more than likely have different absolute time code positions - therefore, one must be offset from the other so that they will match up. Just like there is one master time code for an entire system, each unit only needs one offset value per unit. 01 00 Enable Event List means for a unit to enable execution of events in its list if the appropriate MTC or SMPTE time occurs. 02 00 Disable Event List means for a unit to disable execution of its event list but not to erase it. This facilitates an MTC Event Manager in muting particular devices in order to concentrate on others in a complex system where many events occur simultaneously. 03 00 Clear Event List means for a unit to erase its entire event list. 04 00 System Stop refers to a time when the unit may shut down. This serves as a protection against Event Starts without matching Event Stops, tape machines running past the end of the reel, and so on. 05 00 Event List Request is sent by a master to an MTC peripheral. If the device ID (Channel Number) matches that of the peripheral, the peripheral responds by transmitting its entire cue list as a sequence of Set Up Messages, starting from the SMPTE time indicated in the Event List Request message. 01/02 Punch In and Punch Out refer to the enabling and disabling of record mode on a unit. The Event Number refers to the track to be recorded. Multiple punch in/punch out points (and any of the other event types below) may be specified by sending multiple Set-Up messages with different times. 03/04 Delete Punch In or Out deletes the matching point (time and event number) from the Cue List. 05/06 Event Start and Stop refer to the running or playback of an event, and imply that a large sequence of events or a continuous event is to be started or stopped. The event number refers to which event on the targeted slave is to be played. A single event (ie. playback of a specific sample, a fader movement on an automated console, etc.) may occur several times throughout a given list of cues. These events will be represented by the same event number, with different Start and Stop times. 07/08 Event Start and Stop with Additional Information refer to an event (as above) with additional parameters transmitted in the Set Up message between the Time and EOX. The additional parameters may take the form of an effects unit's internal parameters, the volume level of a sound effect, etc. See below for a description of additional information. 09/0A Delete Event Start/Stop means to delete the matching (event number and time) event (with or without additional information) from the Cue List. 0B Cue Point refers to individual event occurences, such as marking "hit" points for sound effects, reference points for editing, and so on. Each Cue number may be assigned to a specific reaction, such as a specific one-shot sound event (as opposed to a continuous event, which is handled by Start/Stop). A single cue may occur several times throughout a given list of cues. These events will be represented by the same event number, with different Start and Stop times. 0C Cue Point with Additional Information is exactly like Event Start/Stop with Additional Information, except that the event represents a Cue Point rather than a Start/Stop Point. 0D Delete Cue Point means to Delete the matching (event number and time) Cue Event with or without additional information from the Cue List. 0E Event Name in Additional Information. This merely assigns a name to a given event number. It is for human logging purposes. See Additional Information description. Event Time This is the SMPTE/MIDI Time Code time at which the given event is supposed to occur. Actual time is in 1/100th frame resoultion, for those units capable of handling bits or some other form of sub-frame resolution, and should otherwise be self-explanatory. Event Number This is a fourteen-bit value, enabling 16,384 of each of the above types to be individually addressed. "sl" is the 7 LS bits, and "sm" is the 7 MS bits. Additional Information description Additional information consists of a nibblized MIDI data stream, LS nibble first. The exception is Set-Up Type OE, where the additional information is nibblized ASCII, LS nibble first. An ASCII newline is accomplished by sending CR and LF in the ASCII. CR alone functions solely as a carriage return, and LF alone functions solely as a Line-Feed. For example, a MIDI Note On message such as 91 46 7F would be nibblized and sent as 01 09 06 04 0F 07. In this way, any device can decode any message regardless of who it was intended for. Device-specific messages should be sent as nibblized MIDI System Exclusive messages. Potential Problems There is a possible problem with MIDI merger boxes improperly handling the F1 message, since they do not currently know how many bytes are following. However, in typical MIDI Time Code systems, we do not anticipate applications where the MIDI Time Code must be merged with other MIDI signals occuring at the same time. Please note that there is plenty of room for additional set-up types, etc., to cover unanticipated situations and configurations. It is recommended that each MTC peripheral power up with its MIDI Manufacturer's System Exclusive ID number as its default channel/device ID. Obviously, it would be preferable to allow the user to change this number from the device's front panel, so that several peripherals from the same manufacturer may have unique IDs within the same MTC system. Signal Path Summary Data sent between the Master Time Code Source (which may be, for example, a Multitrack Tape Deck with a SMPTE Synchronizer) and the MIDI Time Code Converter is always SMPTE Time Code. Data sent from the MIDI Time Code Converter to the Master Control/Cue Sheet (note that this may be a MTC-equipped tape deck or mixing console as well as a cue-sheet) is always MIDI Time Code. The specific MIDI Time Code messages which are used depend on the current operating mode, as explained below: Play Mode: The Master Time Code Source (tape deck) is in normal PLAY MODE at normal or vari-speed rates. The MIDI Time Code Converter is transmitting Quarter Frame ("F1") messages to the Master Control/Cue Sheet. The frame messages are in ASCENDING order, starting with "F1 0X" and ending with "F1 7X". If the tape machine is capable of play mode in REVERSE, then the frame messages will be transmitted in REVERSE sequence, starting with "F1 7X" and ending with "F1 0X". Cue Mode: The Master Time Code Source is being "rocked", or "cued" by hand. The tape is still contacting the playback head so that the listener can cue, or preview the contents of the tape slowly. The MIDI Time Code Converter is transmitting FRAME ("F1") messages to the Master Control/Cue Sheet. If the tape is being played in the FORWARD direction, the frame messages are sent in ASCENDING order, starting with "F1 0X" and ending with "F1 7X". If the tape machine is played in the REVERSE direction, then the frame messages will be transmitted in REVERSE sequence, starting with "F1 7X" and ending with "F1 0X". Because the tape is being moved by hand in Cue Mode, the tape direction can change quickly and often. The order of the Frame Message sequence must change along with the tape direction. Fast-Forward/Rewind Mode: In this mode, the tape is in a high-speed wind or rewind, and is not touching the playback head. No "cueing" of the taped material is going on. Since this is a "search" mode, synchronization of the Master Control/Cue Sheet is not as important as in the Play or Cue Mode. Thus, in this mode, the MIDI Time Code Converter only needs to send a "Full Message" every so often to the Cue Sheet. This acts as a rough indicator of the Master's position. The SMPTE time indicated by the "Full Message" actually takes effect upon the reception of the next "F1" quarter frame message (when "Play Mode" has resumed). Shuttle Mode: This is just another expression for "Fast-Forward/Rewind Mode". References and Credits SMPTE 12M (ANSI V98.12M-1981). MIDI Time Code specification created by Chris Meyer and Evan Brooks of Digidesign. Thanks to Stanley Jungleib of Sequential for additional text. Also many thanks to all of the MMA and JMSC members for their suggestions and contributions to the spec.