There are many features that help you to manage large networks containing more than one Helix Universal Server. This chapter covers content caching, server redundancy, and proxies.
Normally, if an RTSP connection between RealOne Player and Helix Universal Server version 9.0 breaks, RealOne Player attempts to reconnect to the same server. With a redundant server setup, Helix Universal Server sends RealOne Player a list of alternate servers when it sets up the RTSP control channel. If a break occurs, RealOne Player uses its alternate list to connect to another Helix Universal Server that has the same content. If multiple alternates are available, RealOne Player selects one randomly. The following illustration depicts a RealOne Player failing over to an alternate Helix Universal Server.
| Note: The redundant server feature works only with RealOne Player and later. It does not function with earlier versions of RealPlayer, or any other media players. |
There are three criteria for setting up redundant servers. You need to ensure that the same content is available on each server, and that each server defines a list of redundant servers, along with the rules for when to use them.
Generally, all servers in a redundant cluster must offer the same content, whether on-demand or live. If a redundant server offers only some of the primary server's content, you'll need to set up rules on the primary server to define exactly which failures prompt RealOne Player to reconnect to the alternate server.
| For More Information: You can use the content caching feature, described in "Content Caching", to distribute on-demand content between servers. |
Each Helix Universal Server defines a list of servers available for redundant failover. Typically, each Helix Universal Server in a redundant cluster identifies the other servers in the cluster. Helix Universal Server "A" fails over to Helix Universal Server "B" and Helix Universal Server "C," for example, while Helix Universal Server "B" fails over to servers "A" and "C". If connections to Helix Universal Server "A" fail, each disconnected RealOne Player chooses server "B" or "C" at random. This helps to balance the load if a heavily-used server becomes completely unavailable.
It is not necessary for each server to fail over to every other server in the cluster, however. Helix Universal Server "A" might fail over to Helix Universal Server "B", which fails over to Helix Universal Server "C", which fails over to Helix Universal Server "A". Because you define each server's failover list separately, you have a lot of flexibility over how to set up your redundant cluster.
On each Helix Universal Server in a redundant cluster, you create rules to designate which alternate servers are used when a content stream fails. Because rules are attached to mount points, you can have URLs that use different mount points fail over to different servers, or not fail over at all.
Helix Universal Server searches for and applies rules in an assigned order. For
this reason, you should put longer, more specific rules first. Generally, the
shorter the rule, the broader its scope. For example, assigning the root mount
point (/) to an alternate would result in all disconnects for on-demand content
being redirected to that alternate.
The rule system also allows you to forbid the use of the redundant servers feature on a path-by-path basis. When you exclude a path, Helix Universal Server does not pass a list of alternate servers to RealOne Player while setting up the RTSP control channel. If RealOne Player experiences a disconnect when playing that content, it tries to reconnect to the same Helix Universal Server.
Note:
If you are use the ad serving extension, add the /smilgen/
mount point to the exclude path list. The redundant servers
feature does not work with dynamically generated ad content
that appears under this mount point. For more information
on dynamically generated ad content, see Chapter 15.
|
The following procedure explains how to set up a redundant server list. Carry out this procedure on each server in your redundant cluster.
| To set up an alternate server: |
/ |
/smilgen/ |
Content caching enables you to transfer on-demand streaming media content in any format between two or more Helix Universal Servers. This dynamic distribution improves playback quality by propagating the content closer to the viewer. It also reduces delivery cost by caching content at the network's edge. Distributing content is beneficial if you have large numbers of centrally located, on-demand media files, as well as a number of Helix Universal Servers distributed across your network.
| Note: Caching on servers and proxies works basically the same, but the usage is different. A proxy is used to get through a firewall. A server caches only static content, and works as described in this section. Caching works for all data types. However, if you cache a SMIL file, the content caching feature does not parse the file and cache the clips used in the SMIL presentation. |
The following sections provide an overview of the content caching feature.
In a caching network, you typically manage the bulk of your on-demand media from one or more Helix Universal Servers configured as caching publishers. Other Helix Universal Servers, called caching subscribers, contact the publisher for on-demand content. Each subscriber maintains a portion of physical disk space for caching on-demand content originally deployed on the publisher.
When a media player requests on-demand content from a subscriber, the subscriber attempts to fulfill the request from its cache. If the content is not cached, the subscriber forwards the request to the publisher. As the publisher fulfills the request, the subscriber loads the content into its cache, and streams it to the media player. The following illustration shows a cluster of publishers behind a load balancing solution, delivering content to subscribers in different locations.
RealNetworks recommends that you implement a load balancing solution for Helix Universal Servers acting as publishers. The minimum requirement is DNS rotation, sometimes known as round robin DNS. This technique involves associating multiple IP addresses to a single host name in the DNS record of the host. When the DNS server receives a request for the host name, it rotates through the IP addresses. Thus, the load for the host name is distributed between the IP addresses. Load balancing solutions often use a virtual IP address scheme.
| For More Information: For recommendations about placing a cluster of Helix Universal Servers behind a virtual IP address, see "Working With A Virtual IP Address". |
Once publishers and subscribers are set up, content caching requires little administration. Initial setup can include preloading the cache. Subscribers then prune their caches with a "least recently used" (LRU) mechanism, which removes first those files that have not been requested for the longest time when the cache fills. This way, popular content remains readily available. Removing content from the publisher stops the content from being streamed throughout the network. A subscriber streams only the content that resides on the publisher, even if the content is still available in the subscriber's cache.
Only trusted subscriber Helix Universal Servers may cache content from the publisher. During installation, Helix Universal Server sets up a content caching realm used on the publisher. The person who administers the publisher adds a unique user name and password to the content caching realm, and distributes these credentials to any subscribers set up to access the publisher. In addition to maintaining a secure channel between publisher and subscriber, the administrator of the publisher retains some control over the files served by the subscriber, even if those files already exist in the subscriber's cache.
| For More Information: For information on the search logic between subscriber and publisher, see "Search Logic for Content Caching Rules". |
Content caching ensures that only those portions of media files needed to fulfill a request are transferred from the publisher to the subscriber. In the case of SureStream media requests, only the byte ranges corresponding to the client-subscribed bit rates are transferred. In other words, when a client requests a SureStream clip from the subscriber, the subscriber determines the client bandwidth and requests from the publisher only the portion of the clip that best matches the request. This ensures that the connectivity between the content subscriber and content publisher is used efficiently, and that only requested data resides in the cache. If another client requests content that a subscriber has just cached, but at a different rate, the subscriber adds the new bit rate data to the file.
| Note: If the subscriber administrator preloads a SureStream clip, however, the entire file is loaded in the cache. |
Links to cached content use the same structure as links to regular on-demand content, with the following exceptions:
| For More Information: For more information, as well as specific examples of links, see the section above, "Defining Subscriber Rules". |
There is little to set up on a Helix Universal Server content caching publisher.
You must set up authentication. You must forward the user names and
passwords of the SecureCDist realm to the subscriber administrator, and
indicate the publisher's RTSP port and content mount points. This is so
subscribers can create rules for publishers to use when caching content.
| For More Information: For more on rules, see "Defining Subscriber Rules". |
During installation, Helix Universal Server sets up a default content caching
realm, creates a database of users, and populates the database with a default
user name and password, which is the same user name and password used for
initial access to Helix Administrator. You need to distribute a user name and
password for the SecureCDist realm to the administrators of subscriber Helix
Universal Servers for use when setting up subscriber Helix Universal Servers.
| For More Information: See "Setting Up Realms" for information about authentication realms. |
You set up subscriber Helix Universal Servers after you set up the publisher. Only two steps are required:
| Note: As an option, you can also preload the subscriber cache with files from the publisher. See "Manually Populating a Cache". |
Designate content for caching by setting the content caching mount point flag. Content located under this mount point is then available to the subscriber. The subscriber then enters this mount into a subscription rule.
| To set a content caching mount point flag: |
Each subscriber Helix Universal Server must have at least one publisher Helix Universal Server.
| To define the publisher: |
Yes appears for Enable Content Fetching. This enables the content caching feature.554).Content subscriber rules afford you more control over how content is distributed along your network. You can think of each rule as a subdirectory structure on the publisher that appears in request URLs. For example, consider the following URL:
rtsp://subscriber.example.com/subdirectory/myfile.rm |
In this example, /subdirectory/ is meant to be a subdirectory on the publisher
that you add to the subscriber as a rule. When an incoming URL arrives at the
subscriber, Helix Universal Server attempts to match whatever string it finds
immediately following the host name.
In practice, however, if Helix Universal Server would have found a matching file in its cache, it would have immediately checked the publisher to ensure the file was still available. If the file had been removed from the publisher, the subscriber would not serve the file to the player, even though the file resides in cache. In this way, the administrator of the publisher has some control over the content available to subscribers.
For any of this to work, the string that appears in the URL must match a rule on the subscriber. That rule, in turn, must match the subdirectory structure on the publisher. Thus, the administrator on the subscriber must carefully coordinate with the administrator on the publisher. You can use the asterisk (*) as a wildcard character in rules. In fact, Helix Universal Server adds an implicit wildcard to the end of each rule it processes.
Perhaps the easiest method of setting up content caching is not to use any
subdirectory structure at all. Just use the root of the publisher Helix Universal
Server for storage all on-demand content, and then create a "/" rule on the
subscriber. Consider the following example, meant to be typed directly into
RealOne Player:
r |
This URL combined with a content caching rule for "/" allows for the best of both worlds: all incoming URLs to the subscriber are matched against local content, and then run through the content caching regimen.
| To define content subscriber rules: |
Yes appears for Enable Content Fetching. This enables the content caching feature.Yes is selected in the Enable Rule box.The subscriber must have a cache to store files received from the publisher. By default, Helix Universal Server comes configured with a cache of 1000 megabytes. The size of the cache must be at least 11 megabytes. There is no maximum size restriction.
The cache is automatically pruned based on an least recently used (LRU) mechanism. This means that infrequently requested content is first discarded when the cache reaches its maximum size. If you restart Helix Universal Server, the LRU index is rebuilt based the contents of the cache at the time of restart.
Although there is no functionality to prune the cache with Helix Administrator, there are still ways of manipulating cache content. As has been discussed earlier, if you remove content flagged for content caching from the publisher, the subscriber will not serve that content to players, even if the content remains in cache. Eventually the content will be pruned by the LRU mechanism.
A more direct approach would be to simply delete files directly from the physical disk that holds the cache. Never do this while Helix Universal Server is running, however. Helix Universal Server stores cache contents in memory. Thus, deleting files while Helix Universal Server is running will create a consistency problem. Shut down Helix Universal Server and then delete.
| To define the size of the subscriber Helix Universal Server's cache: |
The physical location of the subscriber is the value entered as the base path of
the /fsforcache/ mount point. You can change the base path of this mount
point, but you should not change the name of the mount point itself.
| To define the location of the subscriber Helix Universal Server's cache: |
"RNCache Local File System".Administrators of subscriber Helix Universal Servers can manually make requests for content. This populates the cache with whatever content the administrator requires. One reason to preload content on the subscriber is so that when clients first start to request files, the publisher is not inundated with requests that all must be fulfilled over the content caching network.
When files are preloaded using the Fetch These Clips box described below, Helix Universal Server requests the entire file. In other words, Helix Universal Server requests all bit-rates for pre-loaded SureStream files. Once the files are pre-loaded into the cache, Helix Universal Server treats it like any other file in the cache, pruning based on an LRU mechanism.
| Note: In addition to preloading the cache from the network, you can also create a disk image of the content you want in cache, and then use that image to propagate cache contents from subscriber to subscriber. You should shut down Helix Universal Server beforehand, however. For more information about where Helix Universal Server stores the physical cache on disk, see "Defining the Cache Location". |
| To populate the subscriber cache manually: |
For example, if publisher1 has static content in its /news/ directory that you want propagated to subscribers, you would add a /news/ rule to publisher1. To fetch content from publisher1, you would enter /news/filename in the Fetch These Clips box (where filename substitute the actual names of the files for pre-loading on the subscriber).
| Note: To manually populate a subscriber's cache, content caching subscriber must be completely configured, including having a least one publisher, one rule, and the Enable Content Fetching turned on. For more information, see "Setting up Content Caching Subscribers". |
This section describes the ways in which content caching works with other features. As a general rule, content caching is comparable to on-demand streaming in how it works with other features.
You can use a proxy to cache or split all on-demand and live content hosted on Helix Universal Server. Using proxies optimizes bandwidth by rebroadcasting content on behalf of clients. For most content, this type of optimization poses no problem. However, there may be some broadcasts or on-demand presentations that you do not want cached or split by a proxy. This section discusses how to define directives for on-demand content and live broadcasts that prohibit proxy devices from caching or splitting streaming media.
| For More Information: See Helix Universal Proxy Administration Guide for information on using Helix Universal Proxy. |
A media proxy is generally installed on an intranet or on an Internet Service Provider's (ISP) network. Media players, such as RealOne Player, are configured to use a proxy for all streaming media requests. When a person requests streaming media, the media proxy acts as an intermediary agent, making the request on behalf of the client. The proxy fulfills the request by locally caching on-demand media, or splitting a live broadcast. In this manner, the proxy conserves bandwidth between it and Helix Universal Server.
RealNetworks certifies third-party media proxies with Helix Universal Proxy technology. This certification ensures two things:
Reports for proxy fulfilled media requests can be accessed in real-time or through the access log. Make sure to choose a RealNetworks certified proxy for your network.
To use Helix Universal Proxy, all media players on the network must be configured to request media through the proxy. When the player makes the initial request for media, a two-way TCP control channel is established. The control channel is for the player software and Helix Universal Server to send information to one another, like rewind and pause commands or user names and passwords. Helix Universal Proxy forwards this initial request to the Helix Universal Server on behalf of the player. Helix Universal Server verifies the existence of the requested file, and whether the client is authorized to view it.
Meanwhile, Helix Universal Proxy monitors these control connections, so that it can account for all clients requesting content from the Helix Universal Server. For Helix Universal Proxy, the control channel is an accounting channel. It's the same TCP connection used by the player and server, but Helix Universal Proxy uses it in a different way. The following diagram illustrates multiple players connecting to a Helix Universal Server through Helix Universal Proxy.
With each player connected to Helix Universal Server in this manner, Helix Universal Proxy can begin streaming media. Helix Universal Proxy streams media based on the type of content being requested, and whether the Helix Universal Server administrator has placed any cache directives on content.
If the stream is live, Helix Universal Proxy uses pull splitting to get an initial stream from Helix Universal Server. This stream is used to fulfill the initial RealOne Player request, as well as any additional requests. The source Helix Universal Server sends only a single stream to Helix Universal Proxy. The following diagram illustrates multiple live streams replicated by Helix Universal Proxy.
If the stream is on-demand, Helix Universal Proxy first tries to fill the request from its media cache. If the content is not yet stored in the cache, Helix Universal Proxy pulls the content from the source Helix Universal Server, simultaneously serving the client and filling the cache. The following diagram illustrates requests for multiple on-demand streams fulfilled by Helix Universal Proxy.
For live or on-demand content, you have full control over what content is available to Helix Universal Proxy. If you have cache directives in place, Helix Universal Proxy functions differently than described above, depending on the type of content being restricted.
If cache directives restrict on-demand content, Helix Universal Proxy does not cache content, nor does it fulfill requests for restricted media from its cache. Because the origin Helix Universal Server fulfills requests for restricted content, there is no bandwidth optimization.
If cache directives restrict a mount point for a live broadcast, Helix Universal Proxy does not rebroadcast a single stream pulled from the origin Helix Universal Server. Instead, it pulls a stream to fulfill each client request. Because the origin Helix Universal Server fulfills requests for restricted content, no bandwidth optimization occurs.
Unless you specify otherwise, all material on your Helix Universal Server is available to Helix Universal Proxy. Helix Universal Server has these options for restricting which Helix Universal Proxys can cache streams:
You can restrict paths, files, or mount points that Helix Universal Proxys can utilize. If Helix Universal Server receives a request for material included in the Deny Cache Requests for these Resource Paths list, it streams the file directly to the client rather than allowing it to be cached and re-transmitted. As always, Helix Universal Server records the transaction in the access log, and reports a bytes sent size of 0 bytes in the cached requests log file.
For example, you might choose to prevent material in authenticated content locations from being cached. Or, you might put the path to time-sensitive clips on this list so that it cannot be stored by Helix Universal Proxy.
| Note: Media caching software makes more streams available on your Helix Universal Server. If you limit clips to be cached, you also limit how many clients you can serve. |
| To prevent Helix Universal Proxy from caching material on Helix Universal Server: |
Content directory contained a directory named news, you would add /news to the Deny Cache Requests for these Resource Paths list. If you only wanted to prevent the late-breaking news clip from being cached, you would add the path and the file name instead (/news/breaking.rm). You can indicate that certain Helix Universal Proxys are not allowed to cache any of your material. To do this, you must know the IP address of the machine on which Helix Universal Proxy is installed.
Tip:
To find out which incoming requests are coming from a
Helix Universal Proxy, look in the rmaccess.log file. You can
identify Helix Universal Proxy request with the client_id field of
the rmaccess.log file.
|
| To prevent certain Helix Universal Proxys from making requests: |
Create an access rule for the Helix Universal Proxy you want to restrict. In addition to specifying the IP address, indicate the port number to which access should be denied (usually 554).
| Note: To learn about limiting access to your Helix Universal Server according to the IP address of any other computer, see "Understanding Access Control". |
| To prevent all caching of all material from all clients and Helix Universal Proxys: |
This section describes how Helix Universal Server interacts with Helix Universal Proxy and other media proxy software.
| Helix Universal Server Feature | Notes |
|---|---|
| On-Demand Streaming | You can mark on-demand content as non-cacheable, on a per-file or per-folder basis. Otherwise, all on-demand clips are automatically available to media caching software. |
| Live Broadcasts | You can create a directive for mount points, so that live broadcasts are not split by media caching software. |
| Access Control | You cannot restrict the IP addresses of an individual client computer. However, you can restrict the IP address of the Helix Universal Proxy that is requesting material on behalf of clients. See "Preventing Helix Universal Proxy from Accessing Helix Universal Server". |
| Authentication | Before allowing clips to be cached, Helix Universal Server verifies whether the client is valid. If the requested material is marked as secured, it then performs any necessary authentication checks. Authenticated material can be stored in a Helix Universal Proxy cache, but the client will be authenticated with the source Helix Universal Server every time it tries to access the stored clip. |
| ISP Hosting | All on-demand material served on behalf of ISP-hosted customers can be cached, unless you mark those directories as non-cacheable (see "Preventing Streams from Being Cached or Split"). |
| Monitoring | The Java Monitor will show the IP address of the caching software as it plays a clip. The caching software is not identified as such. Rather, it appears to be a client. |
| Reporting | To find out which incoming requests are coming from a Helix Universal Proxy, look in the rmaccess.log file. You can identify Helix Universal Proxy request with the client_id field of the rmaccess.log file. |
| Ad Streaming | All material served through the ad streaming feature is marked as non-cacheable. There is nothing you can do to prevent this. |
|
|
© 2002 RealNetworks, Inc. All rights reserved.
For more information, visit RealNetworks Click here if the Table of Contents frame is not visible at the left side of your screen. |