description lang="EN" 
BACKGROUND OF THE INVENTION 

 TECHNICAL FIELD 

[0001] The invention relates to the storing and viewing of television program material in a computer environment. More particularly, the invention relates to playing alternate content during high-speed playback of commercials. 

 DESCRIPTION OF THE PRIOR ART 

[0002] The introduction of Digital Video Recorders (DVR) has revolutionized the television recording industry. DVRs store program material onto a storage medium that is easily accessible, reusable, and the recording does not degrade over time as with videotapes. 

[0003] DVRs give the viewer an unprecedented amount of control over how the viewer watches live and recorded television programs. One of the effects of the DVRs is that as viewers watch television programs stored on the DVR's storage medium, they have a tendency to skip over the commercial breaks. 

[0004] Television broadcasters and advertisers have a negative viewpoint of the advent of the DVR. The feeling is that the DVR is destroying any purpose of advertising on the broadcast medium. Primetime no longer exists because viewers are not tied to the broadcaster's schedule. DVRs allow the viewer to easily store all of his desired programs for later viewing. 

[0005] Broadcasters generate their revenue through the value of the commercial spots. Advertisers feel that they are losing their potential viewership "eyes." If advertisers do not believe that there is any value in a primetime commercial spot, then the broadcasters lose large proportions of their revenue. 

[0006] On the other hand, DVR service providers also generate revenue through commercials. There is an extremely large potential market for specialized advertising that is shown directly to the target audience. DVRs have the ability to evaluate the viewer's viewing preferences and make inferences as to the viewer's lifestyle. With this personal information the DVR service providers can directly target the correct audience for advertisers. 

[0007] It would be advantageous to provide a system for modifying commercials for high-speed playback that allows content providers to retain and improve the value of a certain portion of television commercial breaks. It would further be advantageous to provide a system for modifying commercials for high-speed playback that allows service providers to present advertisements to viewers in advertising space that would normally be skipped by the viewers. 

 SUMMARY OF THE INVENTION 

[0008] The invention provides a system for modifying commercials for multi-speed playback. The system allows the content provider or advertiser to retain and improve the value of a certain segment of television commercial breaks. In addition, the invention provides a system that allows the service provider to present advertisements to viewers in advertising space that would normally be skipped by the viewers. 

[0009] The invention detects when a commercial break begins in a broadcast program content stream. If the viewer selects fast-forward playback mode during the commercial break, then the system selects the appropriate alternate content to be played to the viewer. The alternate content can include images, video, and/or audio content. 

[0010] The system can use a play list that identifies channels and times for affected commercials and the alternate content to be played. The system can automatically detect the appropriate commercial either by the existence of an alternate content set for the specific commercial or by some advanced method such as pattern recognition. 

[0011] The system is provided configuration information by the service provider or broadcaster to identify what alternate content will be played, when it will be played and any additional processing tasks required. The information can also contain business data that identifies whether the system is authorized to play the alternate content. 

[0012] More than one set of alternate content for a specific commercial or commercial break can be provided to the system. The system can select which one to display based on a variety of factors such as user characteristics, time or date when played, or random selection. 

[0013] The system can collect data regarding the playing of alternate content for billing purposes. A service provider can charge an advertiser or content provider a fee based on the data. 

[0014] When the system detects that the commercial break has ended, it returns the viewer back to the program material. 

[0015] Other aspects and advantages of the invention will become apparent from the following detailed description in combination with the accompanying drawings, illustrating, by way of example, the principles of the invention. 

 BRIEF DESCRIPTION OF THE DRAWINGS 

[0016] Fig. 1 is a block schematic diagram of an embodiment of a digital video recording system according to the invention; 

[0017] Fig. 2 is a diagram of a screenshot of a user interface menu listing recorded programs according to the invention; 

[0018] Fig. 3 is a diagram of an exemplary remote input device according to the invention; 

[0019] Fig. 4 is a diagram of a screenshot of a program playback mode according to the invention; 

[0020] Fig. 5 is a block schematic diagram showing a viewer's response to a commercial break within a program according to the invention; 

[0021] Fig. 6 is a block schematic diagram showing the important portions of a commercial break according to the invention; 

[0022] Fig. 7 is a block schematic diagram showing the important portions of a commercial according to the invention; 

[0023] Fig. 8 is a block schematic diagram of an embodiment of the invention showing the bookending concept according to the invention; 

[0024] Fig. 9 is a block schematic diagram of high-level task viewpoint of an embodiment of the invention implementing the bookending function according to the invention; 

[0025] Fig. 10 is a block schematic diagram of an embodiment of the invention showing the concept of playing alternate material during high-speed playback of commercial breaks according to the invention; and 

[0026] Fig. 11 is a block schematic diagram of high-level task viewpoint of an embodiment of the invention that plays alternate material during high-speed playback of commercial breaks according to the invention. 

 DETAILED DESCRIPTION OF THE INVENTION 

[0027] The invention is embodied in a system for modifying commercials for multi-speed playback. A system according to the invention allows the content provider or advertiser to retain and improve the value of a certain segment of television commercial breaks. In addition, the invention provides a system that allows a service provider to present advertisements to viewers in advertising space that would normally be skipped by the viewers. 

[0028] Referring to Fig. 1 , time-shifting devices, or DVRs, take as input television broadcast signals from multiple signal sources. One such method is described in U.S. Patent No. 6,233,389 , entitled "Multimedia Time Warping System", owned by the Applicant. The Input Section 101 selects the input source and directs the signal to the Digital Encoder 102. If the signal source is analog, the Digital Encoder 102 converts the analog signal to a digital form,e.g., MPEG2. For digital signals such as digital satellite feeds, the Digital Encoder 102 can directly store the digital signal or convert the digital signal to a different sample rate. 

[0029] Digital signals are stored on a persistent storage device 103,e.g.,a magnetic hard disk, optical disk, or RAM device. Multiple programs are stored on the storage device 103 and are easily accessed by the viewer. The viewer selects the desired program stored on the storage device 103 through the Viewer Interface 104. Live programs that the viewer tunes to are cached on the storage device 103 as the program is broadcast.
The viewer selects a program via an electronic program guide through the Viewer Interface 104. 

[0030] The selected program is accessed from the storage device 103 and decoded into analog or digital form for presentation onto a television set or monitor by the Decode module 105. If the display is a digital monitor, for example, the Decode module 105 converts the stored program into the proper digital format. The Output module 106 presents the Decode module's 105 output into an acceptable signal format (analog or digital) to the viewer's television or monitor. 

[0031] With respect to Figs. 2 and 3 , an embodiment of the invention presents the viewer with a list 201 of the programs stored on the storage device. The viewer highlights the desired program using a remote input device 301. The program is then selected when the viewer presses the select button 302. Each program is randomly accessible; a program may be viewed and deleted irrespective of its recorded sequence. This is unlike videocassette tapes that do not give the viewer the convenience of easily accessing a particular recorded program on a tape or inserting new recordings amidst a populated tape. 

[0032] Additionally, televised, or live, programs are cached on the storage device as the program is being played to the viewer. Programs may be available by simply selecting a channel, either with the remote input device 301, keypad, or by other methods such as an electronic program guide screen. In this case, the entire program or a portion of the program may be recorded and available to the viewer. 

[0033] Referring to Fig. 4 , DVRs allow the viewer to perform many trick play functions 401 on the program being viewed. The viewer can fast forward and rewind at multiple speeds, pause, single step, slow play, etc. through the program material using a remote input device such as the one depicted in Fig. 3 . The viewer uses the play 303, reverse 304, pause 305, fast forward 306, and frame step 307 buttons to perform trick play functions. If the viewer is using a portable DVR or PC-based DVR, the viewer can access those functions via the keyboard or user interface. 

[0034] The invention of time-shifting devices for television programs has also created the opportunity for television viewers to fast-forward through commercial advertisements if the viewer is watching a recorded program, or a live, cached program where the current viewing position is delayed from the current real-time television broadcast. 

[0035] Since there is little economic incentive for viewers to watch commercials in those cases, the value of the space made available for advertising drops precipitously. However, in examining the behavior of viewers, it is apparent that a newly valuable advertising space has been created at the beginning or ending of a commercial break. 

[0036] With respect to Fig. 5 , an embodiment of the invention provides a method for enhancing the viewership of television ads on a DVR by taking into account the viewer's interaction with the inherent operation of the DVR. Whether the technique is a visual fast-forward (TiVo) or fixed-time (e.g.,30 seconds) skip function (ReplayTV), there is a delay from when the viewer realizes 502 that the program material has ended and advertising has begun 501, and when the viewer starts skipping the commercial using the remote input device 503. Several factors are involved in the viewer's response time. The viewer must first recognize 502 that the program material has ended 501. Once the viewer realizes that the event has occurred, he must then press a button 503 on the remote input device. 

[0037] Often, the time span 507 from the end of the program 501 to the point where the viewer has pressed the button 503 is tens of seconds long. Thus, the first few seconds of any commercial break become extremely valuable as an opportunity to get the viewer to NOT skip the commercial break. Advertisers will pay additional money for the opportunity to occupy this space in the commercial breaks. 

[0038] Similarly, when fast-forwarding or skipping, it is typical that the viewer will not be able to precisely skip to the beginning of the program material 505. When examining viewer behavior, most viewers will skip forward 506, then back up 504 into the preceding commercial break a few seconds in order to view all of the program material from the point where it starts 505. Therefore, the last few seconds of a commercial break 508 become extremely valuable, since it is an opportunity to persuade the viewer to rewind into the commercial break and view advertisements. 

[0039] Referring to Fig. 6 , an embodiment of the invention provides a method wherein the first few seconds 601 or last few 602, are carefully authored to provide a "teaser" to entice the viewer to watch multiple commercials during the break 603. This teaser might be a set of images or logos that indicate a commercial relating to that advertiser is present or a menu or short sequence of animations designed to catch the viewer's attention and persuade him to watch the commercial break. 

[0040] With respect to Fig. 7 , advertisers can also place the more important content in the first 702 or last 703 number of seconds of their commercials 701. This content will be able to get the desired message across to the viewer in those seconds. Given this model, content providers are able to charge advertisers a higher rate to place their commercials at the beginning or the end of a commercial break. 

[0041] Further, one can imagine using the CC tagging technology as described in Application No. 09/665,921 entitled "Closed Caption Tagging System" owned by the Applicant, or similar technology such as ATVEF or Wink, to automatically pause the program being watched immediately at the beginning (or end) of the commercial break, with the frame being displayed containing a menu of the advertisements in the commercial break. The viewer might simply skip forward, or choose a particular commercial via a menu-like overlay on top of the broadcast image using the on screen display (OSD). 

[0042] Another method used is described in U.S. Patent No. 6,215,526, filed on November 6, 1998 , entitled "Analog Video Tagging and Encoding System", owned by the Applicant. The network content provider authors an invisible tag at the beginning of the "pod" (the sequence of ad avails) that tells a decoder (such as TiVo's Time Warping System) what all the commercials in the pod are, thereby allowing the decoder to display an instant menu of ads. 

[0043] Further, it is possible using the CC tagging technology, or similar techniques, to cause an "auto-pause" capability. For example, instead of presenting a menu, a short "teaser" image or multi-frame video could introduce an advertisement, and then automatically pause. The viewer might hit "play" to watch the advertisement or fast-forward to skip to the next advertisement, which would present a similar teaser and then pause. With a just few button presses, the viewer could skip the commercial break and start the program, or watch one or more advertisements in the break. 

[0044] Another method to detect commercial breaks includes some signaling with the program stream that the DVR can detect. For example, include data in video line 21 (for analog and digital television broadcasts) or add digital data to a digital stream. 

[0045] Expanding upon the ability to detect the beginning and end of commercial breaks, another embodiment of the invention detects when a viewer fast-forwards through a commercial break. The system plays alternate content while the viewer is fast-forwarding through the commercial break. The system can play several types of alternate content to the viewer. Some examples are:
A fixed image. This could be displayed for the duration of a commercial or commercial break. Fixed images are an effective means of communication (e.g ., billboards and print advertising).
A slide show. If there is enough time during the commercial break, more than one fixed image can be displayed.
A shortened video. For example, if the DVR is designed for 3x playback during fast-forward, a 10 second commercial could replace a 30 second commercial.

[0046] Any of the above options can be accompanied by alternate audio. Alternatively, the audio can be played through the fast-forwarded commercial or commercial break. This gives the DVR provider an alternate means for advertiser revenue and increased viewership through audio. The viewer must turn down the audio level to avoid the alternate audio track. 

[0047] Any combination of the above options may be used, for example, a short video can play after a fixed image or an alternate audio can be played while a fixed image is being displayed. Even more interesting is that an alternate audio may be selected depending on the viewer's demographic and played over a short video. This would allow the advertiser to target audiences using the same video, but different narratives. 

[0048] The above options may be selected by the system based on the speed of playback that the viewer is using. For example, if the viewer is fast-forwarding at 60x playback then a fixed image would be shown because the commercial break would be skipped in a very short time, but if the fast-forward is at 3x playback, then a short video can be shown. 

[0049] The alternate content can either be delivered to the DVR or generated automatically. As described above, the DVR can automatically display the first frame of video in the commercial. This has the advantage of not requiring any additional data. 

[0050] Content that is sent with the video stream, but is not displayed, can be played by the DVR. For example, an MPEG stream can include additional video, audio and/or data streams. The DVR can also play content that has been sent separately by any means available for communication to the DVR. Broadcasters can include alternate content in their program streams and the DVR can play the alternate content depending on configuration parameters. 

[0051] One skilled in the art will readily recognize that although fast-forward playback has been discussed above, the system can also apply the alternate content playback to slow motion or single-step playback modes. 

[0052] One skilled in the art will readily appreciate that the invention is not limited to DVRs, but can also be applied to video content delivered using other mechanisms (e.g., streaming video on a computer) or to audio content delivery systems (e.g., DVR-like devices for radio). 

[0053] Further, one skilled in the art will readily appreciate that, although commercial breaks are mentioned throughout the text, the tagging methods available allow the invention to replace any section of program material with alternate content. For example, a radio or television show may have certain parts of a program that are not suitable for certain parts of the country and may tag the section of the program so the invention can play alternate content in place of the tagged section. When the program material is cached as in DVRs, the system can actually delay the program material until the alternate content is played if the alternate content is longer than the tagged section. 

[0054] As noted above, the invention stores program material on a storage medium. A viewer accesses the programs through a user menu. The viewer selects the desired program by highlighting the appropriate program and pressing the select button. The viewer can also select program material by tuning to a given channel (virtual or physical) and viewing live or delayed (cached) content. Once the program is selected, it is then displayed. 

[0055] Referring to Fig. 8 , "bookending" concerns the display of the program material 802. The program 802 is indexed and retrieved from the Storage Device 801. Before the program 802 is displayed an ad 803 is first displayed before the program is run. Another ad 804 is then played after the program is over. The invention's bookending allows any ad material that is loaded onto the system to be displayed before and/or after any program material. The ads and program material 805 are displayed to the viewer 806. 

[0056] The ads are stored on the Storage Device 801, indexed and, optionally, stored along with the characteristics of each ad. The choice is up to the DVR system or provider as to whether an ad is played before, after, or both before and after any program material. 

[0057] With respect to Fig. 9 , an embodiment of the invention stores program material and ads on the storage device 903. The Viewer Interface module 901 displays a list of stored or currently televised programs to the viewer and accepts the viewer program selections. The Playback Program module 902 finds and extracts the selected program from the Storage Device 903. For currently televised programs, the Playback Program module 902 switches the DVR tuner to the appropriate channel and the televised program is then stored to the Storage Device 903 and concurrently retrieved by the Playback Program module 902 from the Storage Device 903. 

[0058] Bookending ads are selected by the Bookend Program module 904. The Bookend Program module 904 decides whether to display an ad before and/or after the program material - configurable by the DVR service provider. Each ad is selected from the Storage Device 903 by the Bookend Program module 904. 

[0059] The Playback Program module 902 makes a request to the Bookend Program module 904 for an ad to display. The request is made before and after the Playback Program module 902 displays the program material. If an ad is passed to the Playback Program module 902 by the Bookend Program module 904, then it is displayed. Otherwise, the Playback Program module 902 continues on without displaying an ad. 

[0060] Since the DVR system knows the viewer's program preferences (e.g., science fiction, police dramas) and, possibly, the viewer's personal information (e.g., male, age 27, likes photography), ads can be selected by the Bookend Program module 904 based on this information. The Bookend Program module 904 selects ads targeted to the specific audience that the viewer is a part of. This allows the DVR service provider to charge advertisers for targeted advertising which is much more valuable than shotgun advertising that occurs in normal television commercial breaks. 

[0061] Ads are downloaded onto the Storage Device 903 via a modem, Ethernet, recorded at a predetermined time from an input source, etc. by the Download Ads module 905. The DVR can request ads from a central server or record certain groups of ads for the viewer's target audience. 

[0062] Fig. 10 shows a system that plays alternate material during high-speed playback of commercial breaks. The program material is retrieved from the storage device 1001 and is played to the viewer 1006. The program material can be any multimedia program material. As the viewer 1006 views the program material 1002 the viewer recognizes that the commercial break 1003 has started and fast-forwards through the commercial break. The system detects the beginning of the commercial break 1003 and also detects that the viewer 1006 has switched to fast-forward mode. 

[0063] The system retrieves an alternate image, video, and/or audio 1005 from the storage device 1001 and plays the alternate image, video, and/or audio 1005 to the viewer 1006. The system can play extended or display print ad versions of the alternate image, video, and/or audio 1005 upon the viewer's request. When the system detects the end of the commercial break 1003, it discontinues the playback of the alternate image, video, and/or audio 1005 and switches the viewer back to the program material 1004. 

[0064] Referring to Fig. 11 , an embodiment of the invention stores alternate image, video, and/or audio on the Storage Device 1103. The Viewer Interface module 1101 displays a list of stored or currently televised programs (or multimedia programs) to the viewer and accepts the viewer program selections. The Playback Program module 1102 finds and extracts the selected program from the Storage Device 1103. For currently televised programs, the Playback Program module 1102 switches the DVR tuner to the appropriate channel and the televised program is then stored to the Storage Device 1103 and concurrently retrieved by the Playback Program module 1102 from the Storage Device 1103. 

[0065] The Playback Program module 1102 detects when a commercial break begins and if the viewer selects fast-forward playback mode during the commercial break, then the Playback Program module 1102 requests alternate content from the Playback Alternate Content module 1104. 

[0066] There are a variety of ways for the Playback Alternate Content module 1104 to select the appropriate alternate content to be played back. For example, a play list is sent to the system that identifies channels and times for affected commercials. The play list can be delivered on a different channel, via a phone line, a network, or any other means there is to communicate with the system and stored on the Storage Device 1103. 

[0067] Another example is that the Playback Alternate Content module 1104 can automatically detect the appropriate commercial either by the existence of an alternate content set for the specific commercial or by some advanced method such as pattern recognition. The Playback Alternate Content module 1104 uses this data to know that it must process the commercial during fast playback speed. 

[0068] The Playback Alternate Content module 1104 is provided configuration information by the DVR service provider or broadcaster to identify what alternate content will be played, when it will be played and any additional processing tasks required. The information can also contain business data that identifies whether the system is authorized to play the alternate content. 

[0069] The information can contain data such as:
Commercial identification.
Commercial duration. This allows finer calibration on the alternate content and what can be played at certain playback speeds.
Alternate content type.
Alternate content location (where it can be found by the system, for example, as part of the video stream, or where it was stored ahead of time on the DVR).
Alternate content processing.
Rules for when the alternate content is played.
Rules for data collection for billing (described below).

[0070] This information may also be included with the alternate content data. For example, the data may point to a place on the Storage Device 1103 where the alternate content exists. The alternate content data may include commercial duration, type and processing. 

[0071] More than one set of alternate content for a specific commercial or commercial break can be provided to the system. The system can select which one to display based on a variety of factors such as user characteristics, time or date when played, or random selection. As an option, multiple playing may result in different content each time. 

[0072] The system knows the viewer's program preferences (e.g., science fiction, police dramas) and, possibly, the viewer's personal information (e.g., male, age 27, likes photography), alternate content can be selected by the Playback Alternate Content module 1104 based on this information combined with any rules included in the information sent by the DVR service provider. This allows for content targeted to the specific audience that the viewer is a part of. This also allows the DVR service provider to charge advertisers for targeted advertising which is much more valuable than shotgun advertising that occurs in normal television commercial breaks. If the alternate content is content other than ads, the DVR service provider can charge a fee to content providers for viewership of alternate content. Fees can be charged to an advertiser, content provider, or broadcaster based on viewership, targeting to a specific set of users, targeting to a demographic, etc. 

[0073] To support more advanced business models, the system can collect data regarding the playing of alternate content. For example, it could be simple to count how many viewers watched a certain video. This can be used for statistical or for billing purposes. More advanced collection can identify times played and whether commercials were played at full speed. Data can also be generated that would be valuable for lead generation. 

[0074] Once the Playback Alternate Content module 1104 selects the appropriate alternate content from the Storage Device 1103 or from the program stream, it passes the alternate content to the Playback Program module 1102. As described above, the Playback Alternate Content module 1104 can retrieve alternate content from the program stream itself. The alternate content is not normally displayed in the program stream, but is accessible by the system. The alternate content is extracted from the program stream by the Playback Alternate Content module 1104. The Playback Program module 1102 plays the alternate content to the viewer. 

[0075] The viewer can get an extended version of the alternate content by selecting a specific function on remote control,e.g., slow motion. The Playback Program module 1102 notifies the Playback Alternate Content module 1104 to retrieve an extended version of the alternate content. The Playback Alternate Content module 1104 retrieves an extended version of the alternate content from the Storage Device 1103 and passes it to the Play back Program module 1102. The Playback Program module 1102 plays the extended alternate content to the viewer. 

[0076] The viewer can also get a print ad version of the alternate content by selecting a specific function on the remote control,e.g., pause. The print ad version can contain full product information, contact information, local stores, etc. The Playback Program module 1102 notifies the Playback Alternate Content module 1104 to retrieve a print ad version of the alternate content. The Playback Alternate Content module 1104 retrieves a print ad version of the alternate content from the Storage Device 1103 and passes it to the Play back Program module 1102. The Playback Program module 1102 displays the print ad to the viewer. 

[0077] When the Playback Program module 1102 detects that the commercial break has ended, it returns the viewer back to the program material. 

[0078] The Gather Alternate Content Material module 1104 receives alternate content from the DVR service provider via connections such as satellite, cable, WAN, Internet, telephone lines, etc. The Gather Alternate Content Material module stores the alternate content on the Storage Device 1103. Configuration information that pertains to the alternate content is also received by the Gather Alternate Content Material module 1104 and stored on the Storage Device 1103. 

[0079] Alternate content can also be stored remotely on a network accessible storage device and accessed by the Gather Alternate Content Material module 1104 when needed. For example, a DVR can access other networked DVRs that have stored the appropriate alternate content on their local storage devices. 

[0080] One skilled in the art will readily recognize that although broadcast networks have been described above, the system can also apply to other delivery systems such as point-to-point networks and other network topologies. 

[0081] Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the Claims included below. 

