Abstract

A communications system is provided employing two-way integration approach providing access to notifications and actions occurring on unmanaged communication devices through the use of TV. The system includes installation of an application on unmanaged devices providing the ability to access all notifications and actions occurring on the unmanaged devices using TV. The unmanaged device obtains an input directly from TV from an application server which in turn obtains its input primarily from a TV. The input is not limited to the TV only, and can come from other sources.

Claims

1. A method of controlling at least one unmanaged device via a TV controller using a system, the system comprising: A. said TV controller having a first software agent application thereon; B. said at least one unmanaged device having a second software agent application installed thereon, said second software agent application including a circuit breaker module for protecting said at least one unmanaged device in case of failure of execution of a command; C. at least one service registry installed on at least one of said TV controller and said at least one unmanaged device, adapted for discovery of at least one software agent; D. an intermediary messaging server, for communication between said TV controller and said at least one unmanaged device, said intermediate messaging server including: a. an API gateway module adapted to provide abstracted access to functions of said at least one unmanaged device; b. a transport manager module adapted to manage transport protocols available for communication between said TV controller and said at least one unmanaged device and to translate messages between said transport protocols; c. a partner device router adapted to maintain many-to-many relationships between TV controllers and unmanaged devices; and d. a template directory including a message template directory adapted to manage message templates and an application templates generator, E. an address mapping module, mapping static IDs and dynamic addresses of at least one of said TV controller and said at least one managed device; and F. a partner device discovery module adapted to pair said TV controller with said at least one unmanaged device; said method of controlling at least one unmanaged device via a TV controller comprising: A. sending a first registration message from said first software agent to said partner device discovery module, thereby to register said first software agent; B. sending a second registration message from said second software agent to said partner device discovery module, thereby to register said second software agent; C. pairing said first and second software agents, including: a. when said first and second software agents are in direct communication with one another, at said first software agent receiving said second registration message, and at said second software agent receiving said first registration message, thereby to pair said first and second software agents; and b. when said first and second software agents not in direct communication with one another, at said intermediary messaging server, receiving said first and second registration messages, thereby to pair said first and second software agents via said intermediary messaging server; D. creating or updating a record in said address mapping module, said record including a pair of a dynamic address and a static identifier of at least one of said TV controller and said at least one unmanaged device; E. maintaining a first list of services available on said at least one unmanaged device and a second list of services available on said TV controller, including: a. when said first and second software agents are in direct communication, maintaining said first list in said first software agent and maintaining said second list in said second software agent; and b. when said first and second software agents are in indirect communication via said intermediary messaging server, maintaining said first list and said second list in said intermediary messaging server; F. at said second software agent, listening for pre-defined events and monitoring conditions on said at least one unmanaged device; G. upon said second software agent detecting a monitored condition being met, converting said monitored condition into a condition event; H. upon said second software agent identifying a pre-defined event during said listening or creating said condition event by said translating, at said second software agent, generating a message corresponding to said pre-defined event or to said condition event using a specific message template provided by said template directory, and sending a notification, including said message, to said TV controller, said sending said notification including said message including: a. when said first and second software agents are in direct communication, sending said notification including said message directly to said TV controller, and b. when first and second software agents are in indirect communication: i. sending said notification including said message to said intermediary messaging server; ii. at said intermediary messaging server, performing message transformation and protocol translation so that said notification including said message will be suitable for delivery to said TV controller using a transport protocol available to said TV controller, wherein said message transformation is performed using at least one message template provided by said template directory, and said transport protocol translation is performed using said transport manager module; and iii. delivering said notification including said message to said TV controller, following said message transformation, using said transport protocol available to said TV controller; I. at said first software agent, upon receipt of said notification including said message, rendering a visual presentation of said message using presentation details provided in said message and defined by said specific message template, and displaying said visual presentation on a screen associated with said TV controller; J. following said displaying of said visual presentation on said screen, at said first software agent: a. receiving an input from a user in the form of a selected action button; b. translating said selected action button into a command for execution by said unmanaged device; c. generating a command message including said command using a second specific message template provided by said template directory; and d. sending said command message to said unmanaged device, said sending including: i. when said first and second software agents are in direct communication, sending said command message directly to said unmanaged device, and ii. when first and second software agents are in indirect communication: 1. sending said command message to said intermediary messaging server; 2. at said intermediary messaging server, performing message transformation and protocol translation so that said command message will be suitable for delivery to said unmanaged device using a transport protocol available to said unmanaged device, wherein said message transformation is performed using at least one message template provided by said template directory, and said transport protocol translation is performed using said transport manager module; and 3. delivering said command message to said unmanaged device, following said message transformation, using said transport protocol available to said unmanaged device; and K. at said second software agent, receiving said command message, translating said received command message into one or more executable commands, and executing said executable commands.

2. The method of claim 1, further comprising, upon said displaying said visual presentation on said screen, at said first software agent, pausing a video being played on said screen.

3. The method of claim 1, wherein, during execution of said executable commands, said circuit breaker module carries out the steps of: monitoring execution of said executable commands on said unmanaged device for unusual behavior; and upon detection of unusual behavior, terminating execution of said command on said unmanaged device and attempting to return said unmanaged device to a state corresponding to a state of said unmanaged device prior to the attempt to execute said executable commands.

4. The method of claim 1, further comprising: said API gateway providing abstracted access to function of said unmanaged device; and at said second software agent, using said abstracted access to translate said executable commands to a set of specific functions upon processing said message received from said TV controller.

5. The method of claim 1, further comprising following said pairing and during said listening: at said first software agent: receiving as input from a user a request for carrying out an action involving said at least one unmanaged device; translating said request into an executable command to said unmanaged device for carrying out said action; generating a command message including said executable command using a third specific message template provided by said template directory; and sending said command message to said unmanaged device, said sending including: i. when said first and second software agents are in direct communication, sending said command message directly to said unmanaged device, and ii. when first and second software agents are in indirect communication: 1. sending said command message to said intermediary messaging server; 2. at said intermediary messaging server, performing message transformation and protocol translation so that said command message will be suitable for delivery to said unmanaged device using a transport protocol available to said unmanaged device, wherein said message transformation is performed using at least one message template provided by said template directory, and said transport protocol translation is performed using said transport manager module; and 3. delivering said command message to said unmanaged device, following said message transformation, using said transport protocol available to said unmanaged device; and wherein said listening for pre-defined events by said second software agent comprises listening for receipt of said command message from said first software agent, said method further comprising, at said second software agent: receiving said command message; translating said received command message into said executable commands; and executing said executable command, wherein said request for carrying out an action comprises one of a call for retrieval of specific information from services on said unmanaged device or a command to said unmanaged device.

6. The method of claim 5, wherein, during execution of said function of said unmanaged device corresponding to said executable command, said circuit breaker module carries out the steps of: monitoring execution of said executable command on said unmanaged device for unusual behavior; upon detection of unusual behavior, terminating execution of said command on said unmanaged device and attempting to return said unmanaged device to a state corresponding to a state of said unmanaged device prior to the attempt to execute said executable commands.

7. The method of claim 5, further comprising: said API gateway providing abstracted access to function of said unmanaged device; and at said second software agent, using said abstracted access to translate said executable command to a set of specific functions upon processing said message received from said TV controller.

8. The method of claim 1, wherein said pairing comprises at least one of: using said partner device router and said API gateway, pairing said TV controller to multiple unmanaged devices; and using said partner device router and said API gateway, pairing each of said at least one unmanaged devices to multiple TV controllers, wherein said pairing provides consolidated access to the functions of all paired devices and TV controllers.

9. The method of claim 8, wherein said sending a notification, including said message, to said TV controller comprises sending said notification including said message to said multiple TV controllers.

10. The method of claim 8, wherein, when said notification including said message is in response to a previously sent or received message from a specific one of said multiple TV controllers, sending said notification including said message comprises sending said notification including said message to said specific one of said multiple unmanaged devices.

11. The method of claim 8, wherein said sending said command message to said unmanaged device comprises sending said command message to each of said multiple unmanaged devices.

12. The method of claim 8, wherein, when said command message is in response to a previously sent or received message from a specific one of said multiple unmanaged devices, sending said command message comprises sending said command message to said specific one of said multiple unmanaged devices.

13. The method of claim 1, wherein said pairing said first and second software agents, when said first and second software agents not in direct communication with one another, includes: a. at said first software agent, causing a display associated with said TV controller to display a code; b. sending a message from said first software agent to intermediary messaging server for delivery to said second software agent, said message including said code; c. upon receipt of said message including said code by said intermediary messaging server, storing said code in a memory component of said intermediary messaging server, and delivering a shortened message not including said code from said intermediary messaging server to said second software agent; d. at said second software agent, causing a display associated with said unmanaged device to display a visual representation suitable for code entry; e. upon receipt of a user input code in said visual representation suitable for code entry, sending a response message from said second software agent to said intermediary messaging server for delivery to said first software agent, said response message including said user input code; f. at said intermediary messaging server, comparing said stored code with said user input code; and g. upon identifying a match between said stored code and said user input code, delivering from said intermediary messaging server a pairing confirmation message to said first software agent.

14. The method of claim 1, wherein each said message template defines for a corresponding message a message format and message content, including required message fields, data format, and a layout for presentation of said corresponding message on a screen, said layout including a definition of a visual appearance of said corresponding message, on-screen action buttons, and identifiers of actions.

15. The method of claim 1, wherein: said TV controller comprises a set-top box; said first software agent comprises an application provided and installed on said set-top box; said unmanaged device comprises a smart phone; said second software agent comprises an application installed onto said smart phone; said intermediary messaging server comprises an AMS server, including: an address mapper forming part of said AMS server implementing said address mapping module; a first application adapter implementing said partner discovering module; a second application adapter implementing said partner device router module; a third application adapter implementing said service registry functionally associated with a data storage adapter for storing registries in an associated database; a transport manager of said AMS implementing said transport manager and adapted to support necessary transport actions, transport translation, and message transformation; and a fourth application adapter implementing said template directory, functionally associated with said data storage adapter for storing said message templates in said associated database.

16. The method of claim 1, wherein: said sending said first registration message is implemented upon at least one of booting of said TV controller, activation of said first software agent, a change of IP address of said TV controller, and successful pairing of said TV controller; and said sending said second registration message is implemented upon at least one of booting of said unmanaged device, activation of said second software agent, a change of IP address of said unmanaged device, and successful pairing of said unmanaged device.

17. The method of claim 1, wherein said address mapping module is adapted to maintain, for said TV controller and for each of said at least one unmanaged device, an up-to-date record of a pair of a dynamically changing address and a corresponding static identifier, and wherein said sending said notification and said sending said command message include translating a static address of a receiving device into a corresponding said dynamically changing address of said receiving device for delivery of said notification or said command message by said transport manager.

18. The method of claim 1, wherein, when said TV controller and said unmanaged device are in direct communication, a module performing said message transformation is resident on said TV controller and said unmanaged device.

19. The method of claim 1, wherein said template directory includes an application template generator, and wherein said sending said notification and said sending said command message include at least one of: generating at least one mini-application for execution on at least one of said TV controller and said unmanaged device, said mini-application including at least one widget for display on a screen associated with at least one of said TV controller and said unmanaged device, said widget, when presented to a user, provides a definition of a received message as well as a set of on-screen buttons for user selection of a desired command or action; generating instructions for at least one of said TV controller and said unmanaged device to silently perform an action on behalf of the user without interrupting a viewing experience of the user; extending a functionality of said TV controller using functionalities of said unmanaged device; and extending a functionality of said unmanaged device using functionalities of said TV controller.

Description

FIELD OF THE INVENTION

This invention relates to communications systems, and more particularly to communications systems that employ two-way integration approach providing access to notifications and actions occurring on unmanaged communication devices through the use of TV.

BACKGROUND OF THE INVENTION

Cable television service has become a dominant vehicle for the delivery of electronic entertainment content throughout the United States, and in many parts of the world. Modern cable systems, which were originally designed to deliver analog RF television signals now largely deliver encoded digital data at very high speeds over a conventional, low-loss coaxial cable or a combination of fiber-optic and coaxial cables with appropriate interfaces therebetween. These signals are employed in two-way communication between a cable subscriber and the operator.

A common application involving both two-way communications is the delivery of so-called digital cable service. In a digital cable implementation, the subscriber receives broadcast television signals in digital form via MPEG, TCP/IP or another type of communication. The digital signal is typically received at the subscriber's location by a digital set top box. The set top box is, in essence, a small computer that converts received digital signals into NTSC (or another format) signal capable of being displayed on a conventional television. Most set top boxes, in fact, provide a variety of output connectors that deliver the video signal to a television or display monitor in a variety of formats including S-video, composite and component.

A set top box is a networked computing device. Taking advantage of the availability of two-way digital communication over the line, the set to box can act as a portal through which the subscriber can interact with the head-end server and remote network beyond. Practically all set top boxes accommodate a remote control, that transmits IR (infrared) and/or RF (radio frequency) signals to the box to operate its various functions. Typically set top box functions are displayed via menu screens that often emulate the look and feel of a personal computer's graphical user interface (GUI) display. The remote allows the user to scroll through menu items and highlight the item of interest. Remotes often include a conventional cursor that can be moved about the screen via a four-way toggle to more closely match the point-and-click experience of a personal computer.

Many set top boxes and cable providers now provide functioning web browsers that are accessed by the appropriate remote control buttons and/or menu screens that are displayed on the television. These browsers support certain interactive functions allow entered subscriber requests to be delivered from the server's storage or from the Internet.

Furthermore, the use of mobile or cellular telephones is now extremely common and most available telephones support Short Message Service (SMS) or "text" messaging applications and include small graphical user interface (GUI) display screens for creating and reading such messages (as well as other activities, such as picture/video viewing, games and the like). An SMS message (or simply, "SMS") is convenient way to communicate without need of a voice exchange. Moreover, SMS messages can include embedded hyperlinks (also simply termed, "link(s)") that are transmitted back to a service provider and allow responses from an Internet based content provider via the cellular network. These responses can include downloads to the cellular telephone of requested content from the provider. A technique that further facilitates downloads of such content is also desirable.

On the other hand, the prior art communication systems do not have ability to access all notifications and actions occurring on unmanaged communication devices or mobile devices or phones using TV, and to manage such device activities from TV screen.

SUMMARY OF THE INVENTION

One of the main objects of the invention is to provide a system which includes installation of an application on unmanaged devices such as iPhone or Android phones, also commonly referenced to as a mobile device, providing the ability to access all notifications and actions occurring on such mobile devices using TV. The mobile device obtains an input directly from TV from an application server which in turn obtains its input primarily from a TV. It should be noted however that the input is not limited to the TV only, and can come from other sources. For example, an input can come from another unmanaged device, PC, or another TV.

In the case of TV application, the system enables a user having operational TV to access and display anything that comes from an unmanaged device or mobile device or phone on the TV screen. As an example, if a phone receives a call, a large TV screen will be able to display information which is available on the phone. Thus, a user through the use of a TV screen, has an option on how to manage this call, i.e., hang up, send to voice mail, reply with a short pre-programmed message or to create a custom reply message. A user can respond by instructing a phone how to manage the call from the TV screen. If a person using the phone sends SMS or another message, the message will be also displayed on a TV screen, so that a viewer can ignore the message or reply to such message from the TV screen. According to the invention, a user can provide any notification and control any applications that are available on a phone by using a TV screen. This is inclusive of SMS messages, alarms, emails and other applications. As an example, these applications can include eBay, Amazon or other applications controllable by the user from the telephone. If an alarm ringing on the phone, such alarm will be also displayed on the TV providing a viewer with an alarm reminder of a meeting. In case of a calendar alert or meeting alert, the alert which pops up on the phone can be propagated to the TV screen notifying a viewer about forthcoming event. A viewer can reply by using a TV to instruct the phone either to cancel the alert, reply to it, notify or use that application in any shape or form to send out instructions to the phone on how to handle the notification.

The Bi-directional uniqueness of the invention also consists of adding other applications, so TV can control a phone of the user. For example, in case of a commercial appearing on TV for a local dentist, with a phone number being displayed, the viewer can opt up by pressing remote control buttons to save this number to the phone memory. In this application, TV provides the phone with the name of the dentist, so that the dentist information is placed in the phone book through a command generated by the viewer from a TV screen. In case of an advertisement for an event or a movie on TV, a user can opt to create an event entry in a calendar of a phone based on the TV information.

In a similar fashion, any application available on a phone, such as a weather application, a viewer can obtain related information such as weather conditions as a source feed from an unmanaged device or mobile device or phone to TV screen instead of going elsewhere. According to the invention, a phone is a gateway of a TV viewer to the outside world. This is because practically any information available on the phone will become accessible and can be displayed on and be operated from a large screen TV. In case of e-commerce applications, such as eBay or Amazon, upon receiving a notification on the phone, a viewer can also observe the notification on the respective TV screen and is able to respond to such notification, for example to increase a bid or cancel the bid, etc. all through use of the TV screen. The above demonstrates a bi-directional control provided by a system of the invention, wherein TV is completely in control of the telephone activities of a user, so that the phone can deliver substantial amount of new data and information on a TV display. This represents the ability of the system to convey the data from the mobile device and display it on TV screen.

In the system and method of the invention as a first step, the user generates a command/instruction by pressing a button on a remote control unit (RCU). This command is received by the agent on the set top box, wherein the set top box then communicates the command to the agent on the mobile device and asks for example to retrieve the information of interest such as weather conditions, etc.

As to the next step, the agent on the mobile device communicates to the application on the mobile device or retrieves respective data from an external source, such as web services for example. The obtained data is sent back to the set top box agent. The set top box agent presents the data on a TV screen, so the user can observe his or her request. Along with the actual information being retrieved, the agent on the mobile device also sends to the agent on the set top box the instructions on how this data needs to be represented such as colors, fonts, sizes, position on screen, relative of the elements and so on. The instructions also include indication as to what visual elements are interactive and which commands for the applications on the mobile device are linked to each such interactive visual element. The agent on the set top box does not need to figure out the specific way the data needs to be presented. Instead, it uses the received instructions and generates a typical rectangular window on TV screen to display the retrieved information.

For the above purposes, a user needs to download a proprietary enabled application forming a part of the invention on any ubiquitous computing device or smart device such as cell phone, tablet, etc. This application also provides additional functionalities navigation, voice control, etc. to such devices. After downloading this application, the user performs installation of the application at which point also agrees for the application to have access to notifications and other items on the ubiquitous computing devices. Upon completion of the download and installation, the agent of the invention system will be running and monitoring all events on the unmanaged device. The application on the unmanaged device relays all relevant information to set-top box directly or through intermediary messaging server such as AMS, whereas the intermediary messaging server directs the necessary information to the set top box, wherein the agent is enabled and can be instructed on the set top box what needs to be done. The bi-directional communication occurs the same way when the agent on the set top box collects any information it needs from the TV and sends it back to the intermediary messaging server and the server sends a command with specific instructions to an agent on the unmanaged device to proceed with a specific task, such as to create a contact, send an SMS, make a phone call, reject a phone call, etc. This is a bi-directional control.

Software Agent on TV Controller is a software application that listens to a command from the TV whereas said command can be optionally communicated through an intermediary messaging server and generates instructions for the device what to do. So, if the agent can create a contact on the device, it will do so. If the agent has a permission to make a call from the phone, it will provide instruction to the phone to call a specific number (since VoIP network protocol support is not normally existent on TV Controllers, Message Transformation Module as inventive part of the described functionality will perform message translation from TV controller native protocol to VoIP protocol e.g. SIP). In addition, Software Agent hosts Transport Manager, performs registration on Address Mapper and Device Pairing Function.

The Software Agent on Unmanaged Device is a software application associated with the unmanaged device performs the following main functions: Listens and accepts calls incoming from TV Controller, arriving on different transports and routes them to the underlying unmanaged software through Circuit Breaker adapters. Hosts Mobile Services Registry Performs pairing function. Perform device registration against Address Mapper Subscribes to events (subject to applicable granted permission) happening in unmanaged applications.

Inventive nature of the Software Agent running on Unmanaged Device is the way it is integrated with un-managed applications acting as promiscuous event listener. Such listener can run in pro-active mode (i.e. perform certain business logic actions based in inflow of events and signals from unmanaged applications), or be in reactive mode (where all events are collected unmodified and passed to intermediary Application Server or TV Controller for processing). Reverse mode of operation is also possible. Using colloquial language terms, the Software Agent is informed on everything occurring on the unmanaged device and at the same time can modify or act on the unmanaged on behalf of any software component or as substitute of the customer activities on Unmanaged Device.

The unmanaged Device's Software Agent needs to have a permission to modify the unmanaged device operation to get the notifications. Then it communicates the notifications to the paired device and will listen to the respective reactive signaling to perform actions on the unmanaged device (e.g. place a phone call, send a short message, create a contact list or calendar invitation).

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings which are provided to illustrate and not to limit the invention, wherein:

FIG. 1 is a diagram illustrating system architecture without use of intermediary server middleware of one embodiment of the system and method of the invention;

FIG. 2 is a diagram illustrating system architecture with use of intermediary server middleware of another embodiment of the system and method of the invention;

FIG. 3 depicts an example of a functional flow diagram of the system and method of the invention;

FIG. 4 depicts a system diagram of an example of PIN based partner device discovery of the invention;

FIG. 5 is a system diagram of a transport manager functional component of the system and method of the invention;

FIG. 6A is a diagram illustrating relations of the set-top box and mobile device as supported by the system and method of the invention;

FIG. 6B is a diagram illustrating implementation of a partner device router functional component of the system and method of the invention;

FIG. 6C is a diagram illustrating decision flow of the partner device router functional component of the system and method of the invention;

FIG. 7 is a system diagram of one embodiment of server-side implementation of the service registry functional component of the system and method of the invention;

FIG. 8 is a system diagram of one embodiment of client-side (on a mobile device) implementation of the service registry functional component of the system and method of the invention;

FIG. 9 depicts one embodiment of a message flow diagram related to the service registry functional component of the system and method of the invention;

FIG. 10 depicts a system diagram of intermediate messaging server of the system and method of the invention;

FIG. 11 depicts a system diagram of a template directory functional component of the system and method of the invention;

FIG. 12 is a block diagram illustrating pulling information function of the system and method of the invention;

FIG. 13 is a block diagram illustrating accepting invitation function according to the invention;

FIG. 14 is a block diagram illustrating sending to voicemail function according to the invention;

FIG. 15 is a block diagram illustrating one method of accepting incoming call application function according to the invention;

FIG. 16 is a block diagram illustrating another method of accepting incoming call according to the invention;

FIG. 17 is a block diagram illustrating replying to text message application function according to the invention;

FIG. 18 is a block diagram illustrating saving advertiser to phone book function according to the invention;

FIG. 19 is a block diagram illustrating alerts on TV function according to the invention;

FIG. 20 is a block diagram illustrating low power notification function according to the invention;

FIG. 21 is a block diagram illustrating saving TV screenshot function according to the invention; and

FIG. 22 is a block diagram illustrating pulling contact's location function according to the invention.

FIG. 23 is a block diagram illustrating geographical location notification function according to the invention.

FIG. 24 is a block diagram illustrating estimated time of arrival function according to the invention.

FIG. 25 is a block diagram illustrating setting reminder for TV program function according to the invention.

FIG. 26 is a block diagram illustrating paying with cable bill function according to the invention.

FIG. 27 is a block diagram illustrating paying with mobile function according to the invention.

FIG. 28 is a block diagram illustrating confirming payment application according to the invention.

FIG. 29 is a block diagram illustrating searching for context specific information function according to the invention.

FIG. 30 is a block diagram illustrating tagging video content and publishing tag to social networks function according to the invention.

FIG. 31 is a block diagram illustrating incoming call notification with geographical location of the caller function according to the invention.

FIG. 32 is a block diagram illustrating viewing geographical locations of the contacts function according to the invention.

FIG. 33 is a block diagram illustrating recommended content on TV based on mobile browsing history function according to the invention.

FIG. 34 is a block diagram illustrating auction bidding function according to the invention.

FIG. 35 is a block diagram illustrating time to leave notification function according to the invention.

TERMINOLOGY

For the purpose of the present invention, the term "TV Controller" is used to reference to any device capable of (a) communicating with unmanaged devices as set forth below directly or indirectly via intermediary messaging server, (b) delivering video streams and graphical user interface (GUI) to a TV screen and, (c) receiving user input via connected human interface devices (HID) such as remote control unit (RCU), keyboard, mouse, etc. One example of such TV Controller can be a set-top box (STB) which is used as an example TV Controller throughout of this document.

The following devices, but not limited to, comprise the TV Controller: Set-top box (STB): Set-top boxes with two-way connectivity Note: set-top boxes with one way connectivity such as Digital Television Adapter (DTA) do not constitute TV Controller in terms of the present invention due to lack of ability to send messages to unmanaged devices Game console such as: SONY PlayStation series Nintendo game consoles Microsoft XBOX series Other game consoles Media players such as: Apple TV Google Chromecast Amazon Fire TV Android TV based media players from various vendors Televisions built with Samsung SmartTV technology Televisions built with other smart TV technologies Other media players TV-capable and video-capable devices built with technologies such as: Android TV Apple tvOS Gear VR Linux Microsoft HoloLens Microsoft Windows Oculus Rift Steam OS Stream VR Tizen WebGL WebGL ES Other platforms Personal computer (PC) with graphical user interface (GUI) and operating system (OS) such as Microsoft Windows Apple OSX Linux UNIX Other operating systems Smartphone such as Apple iPhone Android phone Windows phone Other smartphones Tablets such as Android tablets Apple iPad Other tablets

The term "STB", or "set-top box", serves the purpose of the example embodiment of the TV Controller; other TV Controllers as set forth above can be used for such embodiments.

For the purpose of the present invention, the term "unmanaged device" is used to reference a user's device paired with and being accessed and controlled from said TV Controller.

The following devices, but not limited to, comprise the unmanaged device: Smartphone such as Apple iPhone Android phone Windows phone Other smartphones Tablets such as Android tablets Apple iPad Other tablets Personal computer (PC) with graphical user interface (GUI) and operation system (OS) such as Microsoft Windows Apple OSX Linux UNIX Other operating systems

The term "mobile device" is used interchangeably with the term "unmanaged device" as set forth above.

The term "smartphone", serves the purpose of the example embodiment of the unmanaged device; other unmanaged devices as set forth above can be used for such embodiments.

The terms "TV Controller" and "unmanaged device" as set forth in definitions above, can reference to the physically similar or identical devices such as a personal computer, a smartphone, or a tablet.

The present invention specifically differentiates TV Controller as a device primarily utilized by the user to consume video content such as TV channels, Video On Demand (VOD) movies, Per-Pay-View (PPV) content, subscription services such as Netflix, Amazon Prime, Hulu, etc., and other video content. TV Controller typically, but not necessarily, connects to a large screen television (TV) for the comfortable consumption of the video content. Some embodiments of the TV Controller such as personal computers and tablets typically have a screen of the size sufficient for personal video consumption in which case connection to a physical television device is optional or not needed.

The present invention further specifically differentiates unmanaged devices as a device primarily used by the user to consumer other non-video related content and perform other activities such as, but not limited to, telephone calls, video calls, text messaging, activity organizers such as to-do lists, calendars, notes, etc., web surfing, GPS-based navigation (General Positioning System), and other activities.

The terms "Service Operator", "Service Provider", "Cable Operator", "Cable Service Provider", "Multi Service Operator", and "Multi Service Provider" are used interchangeably and refer to the entity such as a commercial company that delivers television services to the one or many geographical areas and may also deliver other services such as Internet access, landline telephony, cellular telephony, web hosting, and other services.

The term "ACable" refers to a fictional Cable Service Provider and used as an example of such company.

The term "Unmanaged Application" refers to a software application on the unmanaged device.

DETAILED DESCRIPTION OF THE INVENTION

Diagram of FIG. 1 refers to the deployment architecture, where all communication between set top box and mobile device(s) occurs in a peer-to-peer fashion, without specific usage of the intermediary messaging server. Such architecture is generally applicable to deployments when set top boxes and mobile devices are connected to the same network, which supports broadcast or multicast discovery of devices and services. Set top box and mobile device are both connected to the same local access network (LAN) such as home Wi-Fi network.

Advanced messaging and network protocol translation is supported in the architecture illustrated in the embodiment of FIG. 2. The diagram of FIG. 2 refers to the deployment architecture that uses intermediary messaging server to facilitate communication between mobile device(s) and set top boxes.

Reference is made now to FIG. 1 disclosing the peer-to-peer deployment architecture. In this arrangement, TV Controller device is running Software Agent. Customer communicates with the TV Controller using remote control unit. Software agent is responsible for handling customer input and for rendering customer User Interface on TV Screen. Software Agent is then responsible for choosing a suitable network transport to deliver messages to and from paired Unmanaged Device. Communication between paired devices in the diagram of FIG. 1 is performed directly from device to device. Software Agent running on Unmanaged Device is responsible for the selection of the suitable network transport, sending and receiving messages to an underlying Unmanaged Software applications running on Unmanaged device.

FIG. 2 depicts the deployment architecture that uses intermediary messaging server to facilitate communication between mobile device(s) and set top boxes. In this arrangement, TV Controller device is running Software Agent. Customer communicates with the TV Controller using remote control unit. Software agent is responsible for handling customer input and for rendering customer User Interface on TV Screen. Software Agent is then responsible for choosing a suitable network transport to deliver messages to and from paired Unmanaged Device. Communication between paired devices on a diagram presented in FIG. 1 is performed through intermediary Application Server. Application server can perform message translation and other functions described in the diagram presented in FIG. 3 and described in detail in the next section. Software Agent running on Unmanaged Device is responsible for the selection of the suitable network transport, sending and receiving messages to an underlying Unmanaged Software applications running on Unmanaged device.

Description of the Unified Messaging System

One of the essential aspects of the invention is to provide a combination of the functional service domains and components enabling bi-directional communication, messaging and micro-services aggregation between TV controllers and unmanaged devices in the household. Some functional domains can be provided by the TV controllers and partnering mobile device, without intermediary messaging server. On the other hand, some functional domains, namely message transformation module or micro-services API gateway, can only be used in the deployment architecture using intermediary messaging server such as Zodiac Interactive's Advanced Messaging Solution (AMS). Some aspects of AMS have been disclosed by U.S. Pat. No. 9,037,667 issued to Leon Rivkin. The entire content of this patent is hereby incorporated in its entirety by reference.

FIG. 3 depicts a diagram showing modules and interconnects as well possible examples of message flows illustrating interaction of the components constituting the invention. The message flows discussed in the application are provided mainly for illustrative purposes. Various alternatives of the message flows, not specifically discussed in the application, are within the scope of the invention.

Among the Messaging groups depicted in FIG. 3 diagram are the following: Message Group 1.x--Registration of TV Controller and Unmanaged Device. Message Group 2.x--discloses typical message flow implementing TV Controller and Unmanaged Device pairing process. Message Group 3.x--discloses component interconnections used by TV controller and Unmanaged Devices to make software components on both TV Controller and Unmanaged Device discoverable in fashion similar to micro-services discovery. Message Group 4.x--discloses the abstracted message flow for one of the example use applications enabled by the present invention (i.e. TV Controller requests and receives information from one or more micro services exposed on the Unmanaged Device through pairing). Message Group 1.X: Device Registration

Message group 1.x discloses applications where TV Controller and Unmanaged device create or update registration record in the following situations:

1. On device boot;

2. Software agent activation:

3. IP address change or other network change;

4. Successful completion of the Device Pairing process.

The purpose of the messaging flow is to register in the system current network address information for the device, to ensure reachability by the paired device or intermediary messaging server. Registration record will be persisted by the Address Mapper functional component of the invention. Address Mapper component of the Unified Messaging system may reside on intermediary Messaging Server, such as disclosed by the Rivkin patent, or directly on the TV controller or Unmanaged Device depending on the embodiment of the invention. In the server less deployment option, registration message flow requires the pre-existing device pairing record. Detailed description of this messaging group is also provided in the Address Mapper section of the application.

Message Group 2.X: Expose Available Services Through Services Registry

Message group 2.x provides typical interaction between functional components exposing available micro-services through the Service Registry functional component, which will be disclosed in greater detail in the description of Service Registry component, later in the application. TV Controller, being managed device, in a typical deployment is not a subject to frequent dynamic changes in the exposed micro-services. Therefore, TV Controller Service Registry is typically hosted and provided on the intermediary messaging service and is available as centralized directory. As opposed to TV Controller, services exposed by Unmanaged Devices are subject to frequent changes. Thus, in a typical deployment, Unmanaged Devices Service registry is distributed on the unmanaged devices themselves. API Gateway and Partner Device Router are components of the invention providing abstracted access to the services of the Unmanaged Devices.

As to the message flow, Message group 2.x has only two messages expose managed services and exposed mobile services. The difference between the two is that managed devices typically expose services through centralized Services Registry (message 2) and unmanaged devices typically have distributed Services registry (message 2.1). Such deployment consideration is driven by dynamic and uncontrollable availability of unmanaged software applications.

Message Group 3.X: Typical Device Pairing Process

Messaging group 3.x provides message flow and interaction of the components for one typical PIN challenge/response implementation of the invention.

The following are the initial requirements to the system to initiate the message flow.TV controller is being booted, software components are loaded and network connectivity is available. Message group assumes device registration has been performed against Address Mapper component (typically deployed on the intermediate messaging server). Direct pairing typically uses network broadcast mechanisms. Consumer, TV Controller, TV and Unmanaged Device should be located at the same physical location allowing direct screen visibility.

As to the message flow, all message communications illustrated in FIG. 3 and described below pass through Message Transformation Module, through pluggable Message Transport Managers, which may select a network transport most appropriate for the network conditions and type of the device engaged in the message interchange. All direct communication links in the diagram of FIG. 3. are subject of the above conditions.

In the diagram of FIG. 3 the message flow for the message group 3.X is as follows: (1) Customer invokes Pairing mode on TV controller using RCU (opposite direction from Unmanaged Device to TV Controller is also possible but not substantially different). (2) PIN/Challenge Phrase is displayed on TV ad described by message 3. (3) Software Agent running on TV Controller sends message to Partner Device Discovery Module, (see corresponding FUNCTIONAL COMPONENT section), where persistent record with the certain expiry timeout is created (message 3.1). (4) Customer invokes Pairing mode on Unmanaged Device and enter PIN/Challenge Phrase. Software Agent on the Unmanaged Devices sends verification message to Partner Device Discovery Module (message 3.2), where PIN/Challenge entered on mobile device is verified. (5) Assuming successful validation, Paired Device meta-information is forwarded back by Partner Device Discovery Module to the TV Controller (message 3.3). (6) Software Agent on TV Controller creates confirmation dialog and displays it on TV. (7) Upon confirmation by Customer (message 3.4) message 3.5 is sent to Partner Device Discovery Module to persist device pairing record and to (optionally) update address mapper record.

Message Group 4.X: Typical Device Pairing Process

Message Group 4.X describes message flow through the functional components of the system of the the invention. This message group is an example application, wherein TV Controller pulls some information from one or more of the applications running on one or more of the Unmanaged Devices.

The following are the initial requirements to the system to initiate the flow for the message group 4.X. TV Controller and Unmanaged Device(s) are registered, having Software Agents installed and network connectivity being established. TV Controller is paired to one or more of the Unmanaged Devices participating in the message flow. Both Message Template (for data interchange) and Application Templates are available in the Template Directory registry (see the description of FUNCTIONAL COMPONENT: TEMPLATE DIRECTORY for greater detailed information).

In the diagram of FIG. 3 the message flow for the message 4.X is as follows: Customer invokes one of the applications available through API Gateway (refer to the corresponding FUNCTIONAL COMPONENT section). Pull mobile info (abstracted. Can be anything e.g. weather data from one of the mobile devices) messages 4 is sent to API gateway. Id of the TV Controller is passed as parameter. API Gateway polls Partner Device Router (refer to the corresponding FUNCTIONAL COMPONENT section) to retrieve list of the Paired Devices for the given TV Controller from Partner Device Discovery Module as described in message 4.1. It is assumed that message format does not require transformation, therefore list of the paired devices is passed directly to Template Directory Component. Partner Device Router, may execute additional business logic for the automatic or semi-automatic selection of the paired Unmanaged Device (see the description of PARTNER DEVICE ROUTER functional component).

API Gateway synchronously instructs Template Directory Component (see the description of TEMPLATE DIRECTORY functional component) to generate and push to the TV Controller widget application capable of displaying dynamic listing of the currently paired devices. Widget application is generated by Application template generator and complemented by the list of devices metadata as described in message 4.2. Application and metadata are then pushed back to the Software Agent running on TV Controller as presented in message 4.3.

Customer interacts with TV Controller using RCU and TV Screen and accepts or modifies suggested Paired Device. Message 4.4 is then sent using TV Controller Transport Manager toward Message Transformation Module of the invention (need for message/protocol translation is assumed for the illustrative purposes).

Translated message 4.5 is then sent to the Unmanaged Device Services Registry using Unmanaged Device Transport Manager as network transport.

Service Registry on Unmanaged Device translates call into at least one (or potentially series), or notifications received from the Unmanaged Device Software as described by message 4.6. Circuit Breaker Component (refer to the corresponding FUNCTIONAL COMPONENT section) of the inventive system is used to protect sensitive API calls or break execution sequence quickly if one of the needed micro-services is not responding.

Resulting mobile info returned in message 4.7 is then travelling back to Message Transformation Module (message 4.8) using network transport designated by Unmanaged Device Transport Manager.

Message Transformation Module transforms message (need for transformation assumed for illustrative purposes) and invokes Template Directory Components for retrieving an appropriate Message Template.

Resulting transformed message is sent back to TV Controller Transport Manager as message 4.9 followed by the TV Controller Application template generated by Application Template Generator in Message 4.10. Immediately following the application template push as outlined in message 4.10, Application Template Generator may also send Automatic Pause message 4.11, switching TV Controller into time shift TV recording mode and allowing customer to interact with activation using remote control unit. Live TV channel is put into pause mode.

Transport Manager passes translated message 4.11 to TV Controller Software Agent combines message 4.11 with application template 4.10 and executes the application template, resulting in display of the requested mobile information on TV Screen.

Each of the presented functional service domains is reviewed below in greater details.

Functional Component: Address Mapper

As depicted in at least FIG. 3, Address Mapper is a functional component of the invention which keeps record of the TV Controller or Unmanaged Device IP or fully qualified domain address up to date. In Address Mapper records are normally updated on upon reaching the following events:

1. On booting/rebooting of the device;

2. Software agent activation;

3. IP address change or other network change;

4. Successful completion of the Device Pairing process.

Partner Device Discovery functional component illustrated in FIG. 4, and discussed later in the application, may consult Address Mapper component to retrieve current network address of the Paired device. Information containing up to date network address of the Device can then be passed to Partner Device Router for message routing. The invention does not mandate usage of IP address as mandatory attribute of the registered device. Use of other network addressing schemas (e.g. Dynamic DNS record or various network address translations mappings) are within the scope of the invention.

Functional Component: Partner Device Discovery

FIG. 4 depicts a system diagram of an example of PIN code based partner device discovery functional component. Partner device discovery is relatively simple when both devices are connected to the same Wi-Fi home network. In such case, standard broadcast oriented discovery mechanisms can be used for pairing on the set top box and mobile device. Set top box and mobile device will perform network handshake without the need of registration server.

In practice, such deployments are not frequent, as mobile devices are often connected to cellular network only. In addition, most Cable Service Providers do not allow direct connectivity from the set top boxes to the public Internet (or even home Wi-Fi network) as part of the "walled garden" network where set-top boxes are only allowed to communicate with authorized services typically residing in back-ends and head-ends that are strictly controlled by said service providers. For such cases, intermediary messaging server which terminates messaging to and from set top boxes is necessary for (a) pairing set-top boxes and mobile devices and (b) relying messages between the two.

Several examples of pairing options include: Federated authentication (using Operator or social network login) through services such as JainRain. SAML or OAuth authentication. Simple PIN challenge/response handshake.

As a functional domain, Partner Device Discovery mechanism does not stipulate any particular implementation. Mechanism presented in the system of the invention stipulates primarily creation of the pairing record describing existing device bond, as described in the section explaining message group 2.x.

The diagram of FIG. 4 depicts sequence diagram of one form of the challenge PIN based device discovery. The following are the initial requirements and assumptions for application of the Partner Device Discovery functional component. Both managed and unmanaged devices have network connectivity and have Software Components constituting the invention installed and running. It is assumed customer activates pairing mode on TV controller first, followed by activation mode on Unmanaged Device. Reverse mode of operation is also within the scope of the invention. Invention embodiment using Application Server is assumed. First two messages of FIG. 4 diagram illustrate registration process for TV Controller and Unmanaged Device respectively and are marked as condition for successful pairing.

Referring now specifically to FIG. 4 illustrating the Discovery Flow process. (1) Customer activates Pairing mode on TV controller. Software agent running on TV Controller generates random challenge PIN and displays it on TV. Generated PIN is set to Application Server (see for example AMS server discussed above). PIN has expiry period. (2) Customer activates paring mode on Unmanaged device then reads challenge PIN from TV screen and enters it on Unmanaged device. (3) PIN entered in a previous step is sent to Application Server where it is matched with the unique pin generated by TV controller. If successful match is found, pairing is considered completed. Pairing metadata is sent to the Application server and confirmation message is displayed on TV screen and acknowledged by customer.

Typical partner device discovery will use enriched set of metadata providing information on the unmanaged device or TV Controller itself, as well as ownership association of the mobile device to a certain member of the household. Such association will be used in the intelligent device routing function described below.

Hybrid model of operation, where two devices are paired on a local network and then pairing record is persisted on the intermediary messaging server is also possible. Hybrid model of operation allows devices to retain pairing bond even when set-top box and mobile device are connected to different networks.

Functional Component: Transport Manager

FIG. 5 depicts a system diagram of a transport manager, which is one of the essential functional components of the invention. Among the essential units of Transport manager are the elements discussed below. Transport Policies Rule Engine, which primary function it to define the set of rules declaring priority and usage restrictions of each transport available through transport plugins. (Example: for any trading application, Transport Policies Rule Engine may enforce use of TLS transport. As Described on the diagram TV Controllers and Unmanaged Devices can use various transport adaptors for different purposes. Multiple transport plugins implementing current and future network transports.

One of the essential aspects of the invention is that the transport manager deployed as part of software agent on the unmanaged device, used in combination with the Message Transformation Module, is used to extend functionality of the TV controller and consequently TV viewing experience with functionality which is not normally available on TV. One example of such functionality is voice transports (SIP for VoIP).

Implementation of the transport manager policies and selection of the transport plugins varies significantly depending on the following factors: Service Operator network topology; TV Controller and to a lesser extent Unmanaged Device hardware, software and available network connectivity; and Software Application in scope.

The diagram of FIG. 5 depicts one of the embodiment of how the Transport Manager can be configured and discloses selection of the network and message transports. Other embodiments are within the scope of the invention. One illustration of the functionality of the Transport Manager, as applicable for the Use Application 6 (Save Advertiser to Phone Book) is described later in the application. Hypothetic network constraints of fictional Service Operator ACable and hardware limitations of TV controllers are considered.

The following are the initial requirements to the system to initiate the process flow for application of the Transport Manager functional component. (1) Service Operator "ACable" provides Linear TV services to TV devices connected to the hybrid fiber-optics/coax (HFC) network using TV controllers. (2) ACable TV Controllers do not have broadband connectivity to the network and their connectivity is limited to DAVIC or ALOHA QAM based in-band and out-of-band IP channels. (3) Practical implication of the previous assumption 2 are as follows: The only channel having sufficient bandwidth for delivery of data to TV controller is in-band QAM channel, that is information must be packed into data PID of MPEG transport stream. (4) Out-of-band IP channel of the management controller is constrained and network spike sensitive. (5) Use Application 6 illustrated in FIG. 18 (saving advertiser to a phone book function) assumes potentially extremely high concurrency, if advertisement metadata is to be delivered during live broadcast of special high viewership events. (6) HTTP, QAM inband (publishing to MPEG TS through video multiplexor), QAM carousel out of band publishing are configured and allowed on the Transport Manager.

The following is one embodiment of the Transport selection flow by Transport Manager, which is configured in transport policies matching ACable network design template. 1. For the flow 2 as presented in FIG. 18 (saving advertiser to phone book) Transport Manager select QAM in-band transport (that is advertiser metadata will be packed into data pad of MPEG video stream, to allow concurrent delivery to multiple TV controllers tuned to a channel). 2. Template of the application, generated by Application Template Generator (part of Template Directory (see FIG. 11) is used by the Software Agent to present a dialog on TV screen allowing push of the advertiser contact information to a mobile device, and selection of the target mobile device using Partner Device Router (see FIG. 7). This template is not changing frequently and is not individually customized per TV Controller; therefore, Transport Manager will choose out-of-band QAM carousel transport for delivery of the application template facilitating the flow according to arrow 3 in FIG. 18. 3. The flow according to arrow 5 in FIG. 18, requires real time delivery to display confirmation message, so Transport Manager will choose RUDP transport from TV Controller to Application server to avoid overwhelming extremely constrained DAVIC/ALOHA QAM out of band transport. Message Transformation Module (see FIG. 10) will convert message to HTTP, and Server Transport Manager will deliver request to Unmanaged device using HTTP transport (see the flow according to arrow 5 in FIG. 18). 4. Returned confirmation message (see the flow according to arrow 6, FIG. 18) will be delivered to Application server via HTTP chosen by Unmanaged Device Transport Manager, will be transformed back to UDP packet by Message Transformation Module (see FIG. 10) and send to the TV Controller Transport Manager using RUDP transport. Functional Component: Partner Device Router

FIG. 6A depicts a diagram illustrating relations of the set-top box and mobile device, whereas FIG. 6B illustrates implementation of a partner device router functional component, whereas FIG. 6C illustrates decision flow of the partner device router functional component. One of the essential features of the presented invention is an ability to facilitate intuitive end-point selection and message routing between set top boxes and mobile devices. By nature, relationship between set top boxes and mobile devices in a household is "many to many" (that is each one of the many mobile devices can be paired to one or several or all the multiple set top boxes in a household). For the purpose of the described invention, way of communication between devices may not necessary be unicast messaging. For example, meeting invitation response for family members, responded via remote on TV can be forwarded to all mobile phones in the household).

As illustrated in FIG. 6B, Partner Device Router includes the following functional components: Active Routes Table provides real-time list of the currently established messaging routes between managed and unmanaged paired devices. If there is such existing route (confirmed by the customer) it is assumed to have highest priority for intelligent route suggestion and is being offered first; State Machine as name implies is a component which maintains route session logic for the life-cycle of the messaging route, from initial hand-shake to cleanup. This component ties all other components together; Route Rules Engine/Learning Function is the component, which processes routing rules presented by the Route Parser. In addition, this component takes meta-data from TV controllers and Unmanaged Devices regarding current TV programming being watched, time of the day based usage statistics etc. Additional metadata is being used by decision making logic for preemptive intelligent target device selection, as described below. Rules parser is responsible for parsing of the route rules presented in configuration data.

Diagram 6b, also shows perimeter components providing auxiliary data feeds facilitating routing decisions, such as time based usage data collection feed, TV programming metadata flow, configuration data. Said feeds do not constitute part of this invention and are not described here.

Partner Device Routing management performs the following essential functions of the invention: Suggest Routing to a mobile device, which belongs to a person whose profile is currently active on a set top box. Suggest Routing to a mobile device, which belongs to a person whose metadata profile (e.g. age or social profile) best corresponds to the metadata of the content being watched on TV (e.g. while watching children programming suggest routing to a mobile phone which belongs to an underage member of the household). Time of the day based routing (e.g. do not suggest routing to a device belonging to a child ???? when current time is later than a configurable threshold. Allow group "multicast style" routing of the messages (e.g. sent meeting invite to all mobile devices in a household).

FIG. 6C depicts the diagram illustrating decision flow of the partner device router functional component. This is an example of the decision logic for one Paired Device selection route flows. Example routing flow assumes existing paired devices. If there is active and not expired route between managed and unmanaged device,--Partner Device Router suggests last used device as routing choice; Otherwise, if any member of the household is logged into set top box, router will attempt to look up mobile device belonging to this person and suggest it as routing choice; Otherwise, if obvious time of the date is possible (e.g. prefer device belonging to an underage person during morning hours), Partner Device Router will make such suggestion; Otherwise, if decision is possible based on the video content metadata (e.g. children programming assumes routing to a device which belongs to an underage person), Partner Device Router suggests such route. As the last resort customer is presented with the list of the paired devices to manually choose from. Functional Component: Services Registry

FIG. 7 depicts a system diagram illustrating a typical deployment of the Service Registry component in the deployment architecture with middleware messaging server. Both mobile devices and set top devices, for the scope of the presented invention, can be defined as devices providing multiple micro-services. In its broadest scope, the perceived goal of the invention is establishment of the bi-directional communication between arbitrary number of the devices, discovery of the micro-services provided by each one and consummation of the micro-services by application miming on any given device. Specifically, to entertainment industry--inventive nature of the described mechanism is the proposed method of creating distributed composite micro-services based software combining functionality of managed TV Controllers and unmanaged devices.

In its simplest degraded form, each application, running on either mobile device or set top box, provides one and only service which is consumed by one and only application running on the paired device. Services Registry is functional component of the present invention which exposes collection of the available micro-services and interfaces available on either mobile device or set top box to a paired device.

Since mobile devices are predominantly unmanaged devices, combination of the available services, messaging data models and templates can vary significantly during mobile device lifecycle, therefore natural place for hosting Services Registry for mobile devices is mobile device itself, where repository of the available services is queried by the Mobile Device Software Agent dynamically. On the other hand, Set Top Boxes are managed devices and software changes on them are much less frequent. In a typical deployment architecture which includes intermediary messaging server, Set Top Box Services Registry will be hosted on the middleware server. For some use applications Application Server may also implement mobile device registry cache.

Diagram presented in FIG. 7 shows typical placement of the Services Registry components, showing centralized deployment for Managed Services Registry and distributed placement of the Managed Services Registry. Centralized Managed Services Registry abstracts and aggregates instances of the individual Managed Devices, where Unmanaged Services Registry is accessible directly on multiple Unmanaged Devices. FIG. 7 illustrates a principle of hybrid deployment schema of the Services Registry. Functions of the Services Registry are also described below and illustrated by FIG. 8 and FIG. 9.

In a simple deployment shown in FIG. 1 (server-less model), Services Registry for both mobile devices and set-top boxes resides on the corresponding device. In complex use cases, more than one of the exposed micro-services can be used by the use application. To facilitate micro-services aggregation, Services Registry for both set top box and mobile devices implements circuit-breaker protection design pattern typical for the distributed micro-service oriented architecture. Diagram of FIG. 8 illustrates an example of Service Registry component breakdown for the mobile device, as presented in the invention.

Example Service Registry component design presented in FIG. 8, is provided for the Services Registry on the mobile device. Among the elements of the Service Registry are: Services List allows consumer application to retrieve list of the available services, as well as obtain description metadata for each provided service. Circuit Breaker Adapter is a software design pattern, used to detect failures and provide encapsulation logic of preventing a failure to reoccur constantly. Circuit Breaker can also be used to isolate non-essential failures, when designing a composite best-effort micro-services based application.

As presented, Service Registry exposes two interfaces: Mobile Service list interface allows discovery of unmanaged micro-services. An example of such implementation will be industry standard Universal Description, Discovery, and Integration (UDDI) specification for SOAP web services, coupled with Web Services Description Language (WSDL) mechanism for discovery of SOAP based web services provided by Software Agent on Unmanaged Device (Note: Other technologies can be used as well). Mobile Application Messaging Interface allows consumption of the actual provided micro-services protected by Circuit Breaker Adapter.

FIG. 9 illustrates the message flow for the composite (complex use case) consumption scenario where set top box application requires certain message types from several mobile applications. Sequence diagram presented in FIG. 9. Illustrates use of the Circuit Breaker by API Gateway Component Adapter as part of the Services Registry.

The following are the initial requirements and assumptions for the composite (complex use case) consumption scenarios: Sequence diagram represents hypothetical application consuming more than one available micro-service available as part of Unmanaged Application1 and Unmanaged Application 2 running on Unmanaged Device. Circuit Breaker Adapter is employed to constantly monitor essential (mandatory) services provided by the Unmanaged Applications 1 and 2. Services provided by these applications are assumed to use external data source and may introduce long time-outs. Composite micro-services based Application running on TV controller requires MessageType1 from Unmanaged Application1, MessageType1 AND MessageType2 from Unmanaged Application2, where MessageType3 from Unmanaged Application2 is optional. Sample application manifest provided: <consume> <required> <Application1> <messageType=1/> </Application1> <Application2> <messageType=1/> <messageType=2/> </Application2> </required> <optional> <Application2> <messageType=3/> </Application2> </optional>

</consume>

One of the embodiments of message flow for the composite (complex use case) consumption scenarios is provided below: 1. TV Controller reaches out to API Gateway running on Application Server (see FIG. 10) which is abstracting access to multiple unmanaged applications under same namespace with consumeComposite passing stbMetadata as parameter. 2. API Gateways performs service discovery against Unmanaged Services Registry and discovers services to be consumed (as per example manifest posted above) 3. Mandatory call is being made callApp1 (message Type=1) through Circuit Breaker Adapter to Unmanaged Application 1. Call monitoring performed by Circuit Breaker does not indicate failure, so call is passed through to the application and returns some reply. 4. As shown in FIG. 9 Loop Monitor App2, is indicating failure of the mandatory service, so Circuit Breaker Adapter, breaks execution and returns failure message mandatory Service Not Available through API Gateway to TV Controller immediately, avoiding potentially long time-out period. Error Handling is activated on TV Controller. Functional Component: API Gateway and Message Transformation Module

FIG. 10 depicts a system diagram which includes Application Server providing API Gateway and Message Transformation Modules for complex user cases. For complex use cases where single application on the mobile device consumes services from multiple applications on the set top box, or where single application on the set top box consumes multiple services or messages from multiple applications running on a potentially multiple mobile device, the Application Server such as Zodiac Interactive's AMS may provide API Gateway and Message Translation Module components.

Diagram of FIG. 10 is provided to illustrate which components of the invention are typically deployed on the Application Server, should embodiment of the invention choose to use such server. Interconnects between components match those provided in the detailed diagram of FIG. 3, and messaging and flows between components are not repeated here. Main function of the API gateway in the system of the invention is to provide unified mapping of the available services and routing to the corresponding Service Registries on multiple mobile devices. API gateway may also maintain a set of business rules which define which services available through Services Registry can be combined together or need to be restricted in any way; Message Transformation Module in the system of the invention is responsible for translation of the messages into formats which are not Internet friendly, or require aggregation/translation of information according to the defined business rules before delivery to the target device.

Message Transformation Module allows TV controller to leverage message transports which are not normally available on TV device (e.g. VOIP or SIP protocols for voice call disposition can be supported using Message Transformation Module to support generation and disposition of voice reply from TV Controller).

Functional Component: Template Directory

FIG. 11 depicts a system diagram of a template directory functional component. One of the industry challenges for designing distributed micro-services based software applications which cross managed and unmanaged devices boundaries, as described in the present invention, is maintaining up-to-date repository of the application templates. In the system of the invention, templates consist of two parts: Message data model template for applications running on mobile devices; and Mini application (widget) templates for defining interaction patterns with mobile applications.

Applications running on the mobile devices due to their unmanaged matter are subject to frequent changes in a course of software development lifecycle. Multiple applications may serve similar function but use different data models for messaging (e.g. mobile device can carry both Google Calendar and Apple Calendar applications which can be used to process meeting invitations. Applications can also use different messaging format such as vCal and iCalendar). Applications may also expand their data models or switch to a different messaging format.

Message Template Directory is a repository of the messaging formats for the known mobile applications covered by the inventive solution. Application Template Generator is a repository of the small application objects used as a mini-application providing service invocation dialogs on either set top box or mobile device. For example, application template may describe menu choices, remote control unit (RCU) key mapping and actions for the Incoming Message Use Application described below. Application template object carries both data properties and optionally behavioral data in a form of scripting fragments. Application template may use standard HTML/XML and JavaScript for formatting and actions respectively, or proprietary formats.

Template Directory is a hosting container defining functionality and data models of the applications making subject of this invention. Examples of the essential applications of the Template Directory are provided below. Other applications are within the scope of the invention. Controlling mobile devices with through software agent on TV controller without interrupting viewing experience Extending managed devices functionality with services available on unmanaged devices (location sensors, VoIP and voice call disposition, SMS, weather sensors, microphone etc.) Usage of unmanaged device as a way to personalize TV experience for choosing profile, leveraging unmanaged device using history, etc. Extending functionality of the unmanaged device by leveraging feed of metadata available from TV Controller managed device (e.g. commercial or featured content metadata).

Application Template Generator is a functional component which generates `render-able` an application which is typically executable on TV Controller. Inventive nature of this component is that application is generated on the fly from the template upon arrival of the corresponding message from the unmanaged device and then pushed to TV controller. Reverse mode of Operation (when application is generated and pushed to mobile device) is also supported. In one embodiment of the system of the invention the Template Directory is hosted on a managed service. Template Directory can be hosted on the Application Server or use third party repository for the server-less deployment as illustrated in FIG. 1.

Automatic Pause Option

Whenever the message is displayed on the TV screen about specific event on the mobile device, the user has options to (a) use remote control unit by means of key presses to select a desired option which will be converted to a command to be sent to a mobile device or (b) not use remote control unit and instead perform desired operations directly on the mobile device. In both cases the so Back to patents