ERROR: Restarting the recording after 2 days is not allowed, MaxRestartGap

Report · Feb 07, 2012

I am getting this error when I am stop and restart a stream after only a few minutes. These are for new files that are definitely not 2 days old. Any idea why im getting this error?

ls live160-2/ -lah

total 39M
drwxrwxrwx 2 fms fms  125 Feb  7 13:41 .
drwxrwxrwx 4 fms fms   38 Feb  7 10:30 ..
-rw-rw-rw- 1 fms fms 2.5K Feb  7 13:41 live160-2.bootstrap
-rw-rw-rw- 1 fms fms   16 Feb  7 10:44 live160-2.control
-rw-rw-rw- 1 fms fms  434 Feb  7 10:41 live160-2.meta
-rw-rw-rw- 1 fms fms  39M Feb  7 10:44 live160-2Seg1.f4f
-rw-rw-rw- 1 fms fms 6.5K Feb  7 13:41 live160-2Seg1.f4x

core.00.log

2012-02-07      13:20:15        9058    (w)2611179      Warning from libf4f.so: [Utils] [live160-2] Discarded all queued Media Messages received before first Video Keyframe Message
.       -
2012-02-07      13:20:15        9058    (e)2611178      Error from libf4f.so: [Utils] [live160-2] [live160-2] Restarting the recording after 2.000000 days is not allowed. Stopping the recording. Please change the config parameter <MaxRestartGap> under <Recording> in event.xml if you would like to increase this
.       -
2012-02-07      13:20:15        9058    (e)2611178      Error from libf4f.so: f4fAdaptor::record failed. Error: Failed to push media message.   -
2012-02-07      13:20:15        9058    (e)2611082      Failed to record live160-2.     -
2012-02-07      13:20:15        9058    (e)2611178      Error from libf4f.so: f4fAdaptor::record failed. Error: Failed to push media message.   -

Report · Feb 07, 2012

What encoder are you using for publishing. Please try with making changes in encoder settings to send the absolute timestamps.. For example in FMLE, you can change syncronisation=true in config.ini..

Because, I suspect that your encoder might be sending from 0 timestamp again as you restart the publishing..

Report · Feb 07, 2012

As the error mentioned, you should also (along with enabling ATC) change the maxrestartgap in event.xml to support more than 2 days of un-restarted recording.

Report · Feb 10, 2012

What to set in MaxRestartGap?

Report · Feb 10, 2012

There is no need to set anything explicit for you issue for this tag. You just need to change what Nitin suggested. If you are done with that then it should take care of your issue. Its only when you have content which is 2 days old and then you decide to append that data after restarting recording after 2 days - then you need to change this tag value - otherwise it can remain the same.

Report · Feb 10, 2012

Thank you for fast response. I've set time sync to true, but same error occured after two days.

Now I've set restart interval to 0 days and 47 hours Still not clear how (what) to set for MaxRestartGap tag.

Report · Feb 10, 2012

So now, you must not be getting the recording issue in case you restart the stream within 2 days..right?

In case, you want stream to be recorded in append mode even after republishing after two days.. You may change in your event.xml by adding or modifying the tag

unders <Recording> section.

Keeping value 3, will allow you to restart stream max within 3 days of the gap.

However, in case you your requirement is not to append to the previous data.. just a fresh recording everytime.. you may choose to modify the livepkgr main.asc to change the recordMode from "append" to "" (empty string)

Report · Feb 10, 2012

So unit of MaxRestartGap is days..

Report · Mar 20, 2012

Nitin

You mentioned changing append to and empty string but I did not think it was possible to alter this in the FMS 4.5.1 streaming edition. I am in the position where I do want just want a fresh recording everytime there is a disconnect ( actually I don't want any recording but I understand that some is required for HTTP Live streams).

Currently we have to monitor the streams directory for Bootstrap files that are older than a minute or two and then clear down the contents of the directory to force a new recording. It is essential that we find a better way of dealing with live stream interruptions as they will happen like it or not.

Nitin Wrote;

>unders <Recording> section.

>Keeping value 3, will allow you to restart stream max within 3 days of the gap.

>However, in case you your requirement is not to append to the previous data.. just a fresh recording everytime.. you may choose to modify the livepkgr main.asc to change the >recordMode from "append" to "" (empty string)

I look forward to hearing from you.

Peter

Report · Mar 21, 2012

Nitin said: "...you may choose to modify the livepkgr main.asc to change the >recordMode from "append" to "" (empty string).."

I have changed mine to "record" - is it wrong?

Report · Mar 21, 2012

No its not wrong - but what Nitin suggested in another post of his of using query paramter is more flexible way of doing it. Remember any change in code requires App Reload - which you can avoid using query parameter way.

Report · Mar 21, 2012

Remember that in the Streaming Edition I don't think you can change the Main.asc file so the query parameter is the only way to do this.

Peter

Report · Mar 22, 2012

Ok, i get it, i was only asking if i change to "record" inside server side code in livepkgr - will it work correctly in record mode or not. (after app reload)

Report · Feb 24, 2012

I am still getting the error. Setting the absolute timestamp didn't help.

I am using Adobe FMLE and Wirecast on various platforms and im having the same trouble on both. I've set the logging level on a test server to Verbose and I was able to get a little more info.

In this test, I started a stream at 1:31PM, stopped it, and restarted it 5 minutes later, and it worked. Then I restarted it 20 minutes after and got the MaxRestartGap error.

Feb 24 13:31:17 adbefms Stream[11610]: Info from libf4f.so: [Utils] [atc_test1] Received ATC offset: 0  .
Feb 24 13:31:24 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] processed onMetadata message. Skipping message .
Feb 24 13:31:24 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Pushed another sample .
Feb 24 13:31:24 adbefms last message repeated 468 times
Feb 24 13:31:24 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Completed Fragment #1 .
Feb 24 13:31:30 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Pushed another sample .
Feb 24 13:31:30 adbefms last message repeated 439 times
Feb 24 13:31:30 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Completed Fragment #2 .
Feb 24 13:31:35 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Pushed another sample .
Feb 24 13:31:35 adbefms last message repeated 357 times
Feb 24 13:31:35 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] completeFragment(): Gap in fragment numbers: Expected 3. Got 4. Assuming the rest of the fragments are skipped .
Feb 24 13:31:35 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Completed Fragment #4 .
Feb 24 13:31:35 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] End of stream .
Feb 24 13:34:35 adbefms Stream[11610]: Info from libf4f.so: [Utils] [atc_test1] Received ATC offset: 179951  .
Feb 24 13:34:48 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Pushed another sample .
Feb 24 13:34:48 adbefms last message repeated 441 times
Feb 24 13:34:48 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] completeFragment(): Gap in fragment numbers: Expected 5. Got 52. Assuming the rest of the fragments are skipped .
Feb 24 13:34:48 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Completed Fragment #52 .
Feb 24 13:34:49 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Pushed another sample .
Feb 24 13:34:49 adbefms last message repeated 144 times
Feb 24 13:34:49 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] Completed Fragment #53 .
Feb 24 13:34:49 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] End of stream .
Feb 24 13:52:47 adbefms Stream[11610]: Info from libf4f.so: [Utils] [atc_test1] Received ATC offset: 4294754452  .
Feb 24 13:52:48 adbefms Stream[11610]: Error from libf4f.so: [Utils] [atc_test1] [atc_test1] Restarting the recording after 2.000000 days is not allowed. Stopping the recording. Please change the config parameter under  in event.xml if you would like to increase this .
Feb 24 13:52:48 adbefms Stream[11610]: Error from libf4f.so: f4fAdaptor::record failed. Error: Failed to push media message.
Feb 24 13:52:48 adbefms Stream[11610]: Failed to record atc_test1.
Feb 24 13:52:48 adbefms Stream[11610]: Error from libf4f.so: f4fAdaptor::record failed. Error: Failed to push media message.
Feb 24 13:52:48 adbefms Stream[11610]: Info from libf4f.so: [Fragmenter] [atc_test1] End of stream .

This is the contents of the log in the admin console:

onPublish : atc_test1
Stream name is: atc_test1 and live event is: timecodetest
f4f:atc_test1 - NetStream.Data.Start
f4f:atc_test1 - NetStream.Play.Reset
f4f:atc_test1 - NetStream.Publish.Start
f4f:atc_test1 - NetStream.Record.Start
f4f:atc_test1 - NetStream.Play.UnpublishNotify
onUnpublish : atc_test1
f4f:atc_test1 - NetStream.Record.Stop
f4f:atc_test1 - NetStream.Play.Stop
f4f:atc_test1 - NetStream.Unpublish.Success
onPublish : atc_test1
Stream name is: atc_test1 and live event is: timecodetest
f4f:atc_test1 - NetStream.Data.Start
f4f:atc_test1 - NetStream.Publish.Start
f4f:atc_test1 - NetStream.Play.Reset
f4f:atc_test1 - NetStream.Record.Start
onUnpublish : atc_test1
f4f:atc_test1 - NetStream.Play.UnpublishNotify
f4f:atc_test1 - NetStream.Record.Stop
f4f:atc_test1 - NetStream.Unpublish.Success
onPublish : atc_test1
Stream name is: atc_test1 and live event is: timecodetest
f4f:atc_test1 - NetStream.Data.Start
f4f:atc_test1 - NetStream.Play.Reset
f4f:atc_test1 - NetStream.Publish.Start
f4f:atc_test1 - NetStream.Record.Start
f4f:atc_test1 - NetStream.Record.NoAccess
f4f:atc_test1 - NetStream.Record.Stop
f4f:atc_test1 - NetStream.Record.NoAccess
f4f:atc_test1 - NetStream.Play.UnpublishNotify
onUnpublish : atc_test1
f4f:atc_test1 - NetStream.Play.Stop
f4f:atc_test1 - NetStream.Unpublish.Success

Report · Mar 14, 2012

This is still happening. Anybody else run into this?

Report · Mar 14, 2012

Please share your

1. Application.xml settings in the livepkgr app

2. config.ini in the FMLE.

I will try to repro at my end..

Report · Mar 15, 2012

Application.xml

<Application>
    <StreamManager>
        <Live>
            <!-- When enabled, the server will assume that incoming live streams have -->
            <!-- timestamps that are based on an absolute clock, such as a SMPTE time -->
            <!-- signal contained within the encoder's input source.                  -->
            <!-- Default is false.                                                    -->
            <AssumeAbsoluteTime>true</AssumeAbsoluteTime>
 
 
            <!-- Allow take over so that encoders that go down (for some reason) and  -->
            <!-- and come back up can immediately republish.                          -->
            <PublishTimeout>0</PublishTimeout>
        </Live>
    </StreamManager>
</Application>

config.ini

###########################################################################
# fms.ini contains substitution variables for Flash Media Server          #
# configuration files. Lines beginning with '#' are considered comments.  #
# A substitution variable is in the form =. Everything up to #
# the first '=' is considered the name of the substitution variable, and  #
# everything after the first '=' is considered the substitution value. If #
# you want a substitution variable to have leading or trailing spaces,    #
# enclose the value around double quotes. For example, foo=" bar "        #
###########################################################################

###############################################################
# This section contains configurable parameters in Server.xml #
###############################################################

# Username for server admin
# For example:
#    SERVER.ADMIN_USERNAME = foo
#
SERVER.ADMIN_USERNAME = allan

# IP address and port Flash Media Admin Server should listen on
# For example:
#    SERVER.ADMINSERVER_HOSTPORT = :1111
#
SERVER.ADMINSERVER_HOSTPORT = :1111

# User id in which to run the process (Linux Only)
# For example:
#    SERVER.PROCESS_UID = 500
#
SERVER.PROCESS_UID = 500

# Group id in which to run the process (Linux Only)
# For example:
#    SERVER.PROCESS_GID = 500
#
SERVER.PROCESS_GID = 500

# License key for Flash Media Server
# For example:
#    SERVER.LICENSEINFO = XXXX-XXXX-XXXX-XXXX-XXXX-XXXX
#
SERVER.LICENSEINFO = 

# LIVE_DIR denotes the full path of sample "Live" application's
# folder for storing any live stream recorded by server.
# For example:
#    LIVE_DIR = \applications\live
#
####LIVE_DIR = /opt/adobe/fms/applications/live
LIVE_DIR = /mnt/media/live_recordings/live

# VOD_COMMON_DIR denotes the full path of sample "VOD" application's
# folder for storing onDemand and Progressive Download .flv/.mp3 files.
# File stored in this folder can be streamed and are also PD-able.
# Note : If you are using the default installation of Apache as a webserver,
# and if you modify VOD_COMMON_DIR, please change the document root
# accordingly in httpd.conf.
# For example:
#    VOD_COMMON_DIR = \webroot\vod
#
VOD_COMMON_DIR = /opt/adobe/fms/webroot/vod

# VOD_DIR denotes the full path of sample "VOD" application's
# folder for storing onDemand only .flv/.mp3 files. Files stored in
# this folder are not PD-able
# For example:
#    VOD_DIR = \applications\vod\media
#
####VOD_DIR = /opt/adobe/fms/applications/vod/media
VOD_DIR = /mnt/media/live_recordings/vod/media

# The maximum size of the FLV cache, in megabytes.
# The default is 500MB.
#
SERVER.FLVCACHE_MAXSIZE=500

# Whether to start and stop the included HTTP server along
# with FMS.
#
SERVER.HTTPD_ENABLED = true

################################################################
# This section contains configurable parameters in Adaptor.xml #
################################################################

# IP address and port(s) Flash Media Server should listen on
# For example:
#    ADAPTOR.HOSTPORT = :1935,80
#
ADAPTOR.HOSTPORT = :1935

# IP (address and) port that Flash Media Server should proxy
# unknown HTTP requests to. Leave empty to disable proxying.
# With no address, specifies a localhost port.
# For example:
#    HTTPPROXY.HOST = webfarm.example.com:80
HTTPPROXY.HOST = :8134

#This tag specifies an IP address for the player to use instead of a hostname when
#making the RTMPT connection to FMS. If nothing is specified, FMS will automatically
#determine the IP to use.
#
ADAPTOR.HTTPIDENT2 =


##############################################################
# This section contains configurable parameters in Vhost.xml #
##############################################################

# Application directory for the virtual host
# For example:
#    VHOST.APPSDIR = C:\myapps
#
####VHOST.APPSDIR = /opt/adobe/fms/applications
VHOST.APPSDIR = /mnt/media/live_recordings

####################################################################
# This section contains configurable parameters in Application.xml #
####################################################################

# List of semi-colon delimited paths in which to search for script to load
# For example:
#    APP.JS_SCRIPTLIBPATH = C:\scripts;C:\Program Files\Foo\scripts
#
APP.JS_SCRIPTLIBPATH = /opt/adobe/fms/scriptlib

###############################################################
# This section contains configurable parameters in Logger.xml #
###############################################################

LOGGER.LOGDIR =

####################################################################
# This section contains configurable parameters in Users.xml #
####################################################################

# Enable or disable using HTTP requests to execute admin commands.
# Set to "true" to enable, otherwise it will be disabled.  The
# actual commands permitted for server admin and virtual host admin
# users can be set in Users.xml.

USERS.HTTPCOMMAND_ALLOW = true

Report · Mar 19, 2012

Nitin has asked for config.xml of FMLE and not fms.ini of FMS. You can find config.xml (i think by mistake Nitin put wrong extenstion) for FMLE - you can find the same in <installDir of FMLE>/Conf

Report · Mar 19, 2012

config.xml

<?xml version="1.0" encoding="UTF-16"?>
<flashmedialiveencoder_config>
     <!-- This tag configures FMLE for the case when audio device sends audio samples-->
     <!-- with forward timestamp. In this case FMLE can be configured to send           -->
     <!-- silence audio message between samples that have forward timestamp.               -->
     <filler>
          <!-- Min Forward timestamp duration (in Milli Seconds).                              -->     
          <!-- Default value is 50ms.                                                                 -->     
          <duration>50</duration>
          <!-- Buffersize for the resampler and filler                              -->     
          <!-- Default value is 2000ms. -->
          <buffersize>2000</buffersize>
     </filler>
     
     <!-- This tag configures FMLE for Debug Log writings.                                   -->
     <!-- Debug logs for audio, video, rtmp, etc. are created and can be analyzed     -->
     <!-- for troubleshooting purposes.                                                            -->
     <!-- Logs are created in folder "stats_yyyymmdd_hhmmss" inside MyVideos           -->
     <!-- directory by default. Logs directory path can be changed using "path"          -->
     <!-- tag.                                                                                          -->               
     <logs>
          <!-- Enable/Disable log writings.                                                       -->
          <!-- "true"  to Enable log writings.                                                  -->
          <!-- "false" to Disable.                                                                 -->
          <enable>false</enable>                                                                      
          <!-- Logs Directory path.                                                                 -->
          <!-- For Example, "C:\FMLE Logs".                                                       -->
          <!-- By Default MyVideos is taken.                                                       -->          
          <path></path>     
     </logs>
     <mbrconfig>
          <!-- Multiple Stream synchronization at FMS.                                        -->
          <!-- This feature can be used to synchronize streams from multiple FMLE          -->
          <!-- instances running on multiple machines.                                        -->
          <!-- All the streams are synchronized using a common reference UTC               -->
          <!-- (Coordinated Universal Time) time.                                                  -->
          <!-- All the FMLE stream that needs to be synchronized should be configured     -->
          <!-- with same reference time and the machine on which FMLE instances are       -->
          <!-- running should be synchronized using NTP (Network Time Protocol)            -->
          <!-- for proper stream synchronization at FMS.                                        --> 
          <!-- This feature is disabled by default.                                             -->
          <streamsynchronization>
               <!-- "true" to enable this feature, "false" to disable.                         -->
               <enable>true</enable>
               <!-- A Past UTC reference time.                                                       -->
               <!-- Past reference time should not be earlier than 30 days from the      -->
               <!-- time when encoding is started.                                                  -->
               <!-- If invalid reference time is provided, current month is taken as      -->
               <!-- reference time.                                                                 --> 
               <referencetime>
                    <!-- A past/current month.                                                       -->
                    <!-- 1 for January, 2 for February and so on.                              -->
                    <!-- 0 means use current month.                                                  -->     
                    <month>0</month>
                    <!-- A past/current year.                                                       -->
                    <!-- for Example, 2008,2009 etc.                                             -->
                    <!-- 0 means use current year.                                                  -->
                    <year>0</year>
               </referencetime>
          </streamsynchronization>
     </mbrconfig>
     <rtmpconfig>
          <!-- Maximum RTMP buffer size and buffer duration configuration.               -->
          <!-- This tag can be used to set maximum RTMP buffer size and maximum          -->
          <!-- buffer duration at FMLE side.                                                       -->
          <!-- Sometimes while publishing in adverse network condition (low bandwidth,-->
          <!-- high latency, etc.), the RTMP buffer at FMLE could     grow enormously          -->
          <!-- and cause increase in memory usage (Virtual memory).                         -->
          <!-- Very high memory usage can cause application to become unresponsive     -->
          <!-- or even crash.                                                                           -->
          <!-- Use this tag to set a threshold value for the RTMP buffer.     Once any     -->
          <!-- of the threshold value (maxrtmpbuffersize or maxrtmpbufferlength) is     -->
          <!-- reached, FMLE flushes the RTMP buffer and re-publishes the streams.     -->
          <!-- When the RTMP buffer is flushed, all the data in the buffer is               -->
          <!-- discarded, leading to loss of the bufferd data.                              -->
          <!-- In order to avoid increase in RTMP buffer and prevent the data loss,     -->
          <!-- use AutoAdjust feature. Please refer to help file for AutoAdjust.           -->
          <!-- This feature is enabled by default.                                             -->
          <rtmpbuffercontrol>
               <!-- "true" to enable this feature, "false" to disable."                    -->
               <enable>true</enable>
               <!-- Maximum RTMP buffer size. (in KB)                                             -->
               <!-- This tag specifies how large the RTMP buffer size could grow.           -->
               <!-- default value is 512MB (512,000KB).                                        -->
               <maxrtmpbuffersize>512000</maxrtmpbuffersize>
               <!-- Maximum RTMP buffer Length/duration (in Seconds).                         -->
               <!-- This tag specifies the maximum RTMP buffer length/duration.          -->
               <!-- Use this tag to set the threshold value for the RTMP buffer length.-->
               <!-- Default value is 3600 seconds ( 1 hour).                                   -->          
               <maxrtmpbufferlength>3600</maxrtmpbufferlength>
          </rtmpbuffercontrol>               
     </rtmpconfig>
</flashmedialiveencoder_config>

Report · Mar 20, 2012

Try changing livestream and liveevent name.

I have same issue if i use same name, but if i change the name everything goes fine.

Like there is 'overwrite forbiden'.

Now second part is how to set it up to work with sam stream name.

BR

Vlado

Report · Mar 20, 2012

I have not had any problems using the same names for livestream and

livevent as far as I am aware we have several streams running in

separate livepkgr instances with the same stream and event name.

I will investigate further.

Peter

Report · Mar 20, 2012

I am sorry.. Couldn't get time to investigate this on my side.. Will check if someone in my team can take this up.. Will keep this in my list..

Also, you have valid concerns that we can't change recording mode in streaming edition. All I can say that streaming edition comes with minimal power right now.. But I got your concern and will forward it to my team - how we can improve streaming edition's user experience with livepkgr application.

Thanks for giving us thoughts on improving the FMS.

Report · Mar 20, 2012

Hey, you can do one thing.. in case you want to record in "record" mode rather than "append"

add query parameter adbe-record-mode="record" in the publish stream.

for example publish

livestream?adbe-live-event="liveevent"&adbe-record-mode="record"

Thanks

Report · Mar 21, 2012

Hi,

Is there any documentation on this parametars?

It works now when i add it to my stream, but i dont know what it does.

Will it record whole stream to file, or?

Also,

I have FMS 4.5 Streaming edition installed, but i dont want to record streams.

Where can i chenge this?

BR

Vlado

Report · Mar 21, 2012

In Event.xml in the recording section you can use this in the <Recording> section the number used refers to hours of recordings on disk.

We are not interested in recording so keep it fairly small. I guess if you are utilising the DVR function this will have to big enough to fulfill the DVR needs.

There are other sections in the Application.xml where you can set recording time but they did not seem to have much impact in the streaming edition.

Hope that help

Peter

Adobe Community

ERROR: Restarting the recording after 2 days is not allowed, MaxRestartGap

ls live160-2/ -lah

core.00.log

Application.xml

config.ini

config.xml