Tekkotsu Homepage
Demos
Overview
Downloads
Dev. Resources
Reference
Credits

FileSystemDataSource Class Reference

Manages the loading of a series of files from disk. More...

#include <FileSystemDataSource.h>

Inheritance diagram for FileSystemDataSource:

Detailed Description

Manages the loading of a series of files from disk.

Can handle an index file listing other data files and their timestamps, a directory holding a set of files, or a single explicit file.

Definition at line 17 of file FileSystemDataSource.h.

List of all members.

Classes

struct  DataThread
struct  FileInfo
 stores basic information regarding each file in the queue, including routines for loading to and from disk More...

Public Member Functions

 FileSystemDataSource (LoggedDataDriver &p, const std::string &filter)
 constructor
 ~FileSystemDataSource ()
 destructor
virtual unsigned int nextTimestamp ()
 Returns the simulator time of the next data segment.
virtual const std::string & nextName ()
 Returns a descriptive name of the next data segment for user feedback (e.g. a filename for logged data).
virtual bool advance ()
 Called when the simulator is stepping while paused or advancing a frozen data source, return true if successful, or false if no more data is available.
virtual void registerSource ()
 User hook, called when the data source should claim which outputs it provides feedback (providingOuput()).
virtual bool isRegistered () const
 returns registered
virtual void deregisterSource ()
 User hook, called when the data source should release its claim on outputs with feedback (ignoringOuput()).
virtual void enteringRealtime (const plist::Primitive< double > &simTimeScale)
 User hook, called when the controller is going to be running in realtime mode, which is probably the normal mode you'd expect.
virtual void leavingRealtime (bool)
 User hook, called when leaving realtime mode, which means you have no idea when motionCheck() is going to be called in terms of wall-clock time.
virtual void setFrame (unsigned int f, unsigned int numPreload=2)
 sets the next frame to be sent (e.g. pass 0 to reset to the first frame)
virtual void nextFrame (unsigned int numPreload=2)
 increments curfile to the next frame, preparing (pre-loads) numPreload frames from the new curfile onward
virtual double getLoopTime (bool actual=true) const
 returns the total time taken to loop over the files
virtual void setLoopTime (double t)
 adds time to the final frame to increase total sequence time to t milliseconds
virtual bool usingIndexFile () const
 returns indexed
void clearFiles ()
 empties files
virtual void plistValueTouched (const plist::PrimitiveBase &pl)
virtual void plistValueChanged (const plist::PrimitiveBase &pl)
virtual void loadXML (xmlNode *node)
virtual const std::string & getUsedPath () const
 returns the target path, either path, or parent.path if path is empty
virtual bool loadFileList (bool clearCurrent=true, bool reportMissing=true)
 call this to (re)load the list of available file names from disk

Public Attributes

plist::Primitive< std::string > path
 The directory, data file, or index file from which to load via call to loadFileListFromDirectory(), loadSingleFile(), or loadFileListFromIndex().
plist::Primitive< std::string > filenameFilter
 a regular expression (POSIX.2 extended format) to select which files to load from path, if path is a directory or index file
plist::Primitive< boolloop
 controls whether to restart curfile at the beginning of files when it reaches the end
plist::Primitive< floatframerate
 The sensors updates per second which should be loaded. Note this is limited by the global Sensors.Framerate setting.
plist::Primitive< int > verbose
 Controls how much feedback to give on the console regarding progress.

Protected Types

typedef std::list< FileInfo * > files_t
 type of files, the list of files to load

Protected Member Functions

virtual bool sendData ()=0
 Gives the data referenced by curfile to the simulator.
virtual void doFreeze ()
 user hook for when frozen is set to true; advance() will be called by simulator at user discretion.
virtual void doUnfreeze ()
 user hook for when frozen is set to false; if enteringRealtime() has been called then you should resume sending data.
virtual void resetPoller ()
virtual void loadSingleFile (const std::string &file)
 load a single file
virtual void loadFileListFromDirectory ()
 load a list of files from a directory specified by path
virtual bool loadFileListFromIndex ()
 load a list of files from an index file specified by path
virtual double calcLoopTime () const
 adds up the lifetime of all files, plus initialDelay
virtual void preprepare (const FileSystemDataSource::files_t::iterator &fi)
 calls prepare on fi, dumping diagnostic info if verbose is high enough
virtual void enqueueFile (const std::string &name, double lifetime)
 creates a new entry on files, virtual to allow subclasses to use a FileInfo subclass with more meta data (e.g. see FileSystemImageSource::ImageInfo)

Protected Attributes

FileSystemDataSource::DataThread poller
files_t files
files_t::iterator curfile
 an iterator referencing files -- indicates next file to send
double initialDelay
 milliseconds to wait before sending first frame
double nextTime
 timestamp that curfile should be sent
bool indexed
 true if the file list was specified by an index file
unsigned int freezeTime
 time at which doFreeze was called
const plist::Primitive< double > * timeScale
 simulator's timeScale parameter, so we can register for updates when it changes
LoggedDataDriverparent
 device driver this is a member of
double actualLoopTime
 time it would take to run through all of the loaded frames
double naturalLoopTime
 time it would take to run through all of the frames as set immediately after load (i.e. no setLoopTime())
Thread::Lock lock
 don't be in advance() while modifying file list
bool registered
 synced to registerSource/deregisterSource so we don't start the thread if not actually in use

Static Protected Attributes

static const unsigned int MAX_LOAD = 4000
 maximum number of data elements to try to keep 'active'. If there's more than this in files, we'll only load one at time, and immediately release it afterward

Private Member Functions

 FileSystemDataSource (const FileSystemDataSource &)
 don't call
FileSystemDataSourceoperator= (const FileSystemDataSource &)
 don't call

Member Typedef Documentation

typedef std::list<FileInfo*> files_t [protected]

type of files, the list of files to load

Definition at line 98 of file FileSystemDataSource.h.


Constructor & Destructor Documentation

FileSystemDataSource ( LoggedDataDriver p,
const std::string &  filter 
)

constructor

Definition at line 20 of file FileSystemDataSource.h.

destructor

Definition at line 21 of file FileSystemDataSource.cc.

FileSystemDataSource ( const FileSystemDataSource  )  [private]

don't call


Member Function Documentation

bool advance (  )  [virtual]

Called when the simulator is stepping while paused or advancing a frozen data source, return true if successful, or false if no more data is available.

Implements DataSource.

Definition at line 41 of file FileSystemDataSource.cc.

Referenced by DataThread::run(), and setFrame().

double calcLoopTime (  )  const [protected, virtual]

adds up the lifetime of all files, plus initialDelay

Definition at line 257 of file FileSystemDataSource.cc.

Referenced by loadFileList().

void clearFiles (  ) 
void deregisterSource (  )  [virtual]

User hook, called when the data source should release its claim on outputs with feedback (ignoringOuput()).

It would be wise to call this from your destructor as well.

Reimplemented from DataSource.

Reimplemented in FileSystemSensorSource.

Definition at line 96 of file FileSystemDataSource.cc.

void doFreeze (  )  [protected, virtual]

user hook for when frozen is set to true; advance() will be called by simulator at user discretion.

Reimplemented from DataSource.

Definition at line 120 of file FileSystemDataSource.cc.

void doUnfreeze (  )  [protected, virtual]

user hook for when frozen is set to false; if enteringRealtime() has been called then you should resume sending data.

Reimplemented from DataSource.

Definition at line 124 of file FileSystemDataSource.cc.

virtual void enqueueFile ( const std::string &  name,
double  lifetime 
) [protected, virtual]

creates a new entry on files, virtual to allow subclasses to use a FileInfo subclass with more meta data (e.g. see FileSystemImageSource::ImageInfo)

Reimplemented in FileSystemImageSource, and FileSystemSensorSource.

Definition at line 168 of file FileSystemDataSource.h.

Referenced by loadFileListFromDirectory(), loadFileListFromIndex(), and loadSingleFile().

void enteringRealtime ( const plist::Primitive< double > &   )  [virtual]

User hook, called when the controller is going to be running in realtime mode, which is probably the normal mode you'd expect.

You might be in realtime mode, but a debugger breakpoint will still pause things, or thread scheduling could hiccup, so try to be robust.
The argument is a reference to SharedGlobals::timeScale, so the data source can subscribe to changes in simulation speed if it can use that information. (We avoid direct dependency on the tekkotsu simulator so this code can be reused for other tools too.)

Reimplemented from DataSource.

Definition at line 108 of file FileSystemDataSource.cc.

virtual double getLoopTime ( bool  actual = true  )  const [virtual]

returns the total time taken to loop over the files

Parameters:
actual if false, will return the "natural" time of a loop, ignoring setLoopTime() effects

Definition at line 54 of file FileSystemDataSource.h.

Referenced by advance(), nextFrame(), LoggedDataDriver::plistValueChanged(), and setLoopTime().

const std::string & getUsedPath (  )  const [virtual]

returns the target path, either path, or parent.path if path is empty

Definition at line 144 of file FileSystemDataSource.cc.

Referenced by loadFileList(), loadFileListFromDirectory(), loadFileListFromIndex(), and LoggedDataDriver::plistValueChanged().

virtual bool isRegistered (  )  const [virtual]

returns registered

Definition at line 40 of file FileSystemDataSource.h.

Referenced by LoggedDataDriver::plistValueChanged().

void leavingRealtime ( bool   )  [virtual]

User hook, called when leaving realtime mode, which means you have no idea when motionCheck() is going to be called in terms of wall-clock time.

Argument set to true if entering full speed mode, which indicates everything should run at full native "frame rate", and may indicate more data will be processed than normal, CPU speed permitting. However, if false, almost certainly indicates updates will be sparse, trigger by user 'step' commands. May be called multiple times if changing between full-speed mode and paused

A non-realtime mode might be triggered if the user wants to pause the simulator/controller to step through something... No guarantees though! The debugger might catch a breakpoint and stop things, and this won't be called!

Reimplemented from DataSource.

Definition at line 112 of file FileSystemDataSource.cc.

Referenced by deregisterSource(), and ~FileSystemDataSource().

bool loadFileList ( bool  clearCurrent = true,
bool  reportMissing = true 
) [virtual]

call this to (re)load the list of available file names from disk

If clearCurrent is set, then the current file list will be cleared; otherwise, the loaded files will be appended to the current queue

Definition at line 146 of file FileSystemDataSource.cc.

Referenced by LoggedDataDriver::plistValueChanged(), plistValueChanged(), and plistValueTouched().

void loadFileListFromDirectory (  )  [protected, virtual]

load a list of files from a directory specified by path

Definition at line 338 of file FileSystemDataSource.cc.

Referenced by loadFileList().

bool loadFileListFromIndex (  )  [protected, virtual]

load a list of files from an index file specified by path

This supports either the format produced by VisionGUI, or a simplier 'filename [<tab> time]\n' format, where if time is unspecified, the frame's time is incremented by the framerate from the previously listed file. Filenames should either be either absolute paths or relative to the directory which contains the index file.

Definition at line 401 of file FileSystemDataSource.cc.

Referenced by loadFileList().

void loadSingleFile ( const std::string &  file  )  [protected, virtual]

load a single file

Definition at line 332 of file FileSystemDataSource.cc.

Referenced by loadFileList().

void loadXML ( xmlNode node  )  [virtual]

Definition at line 321 of file FileSystemDataSource.cc.

void nextFrame ( unsigned int  numPreload = 2  )  [virtual]

increments curfile to the next frame, preparing (pre-loads) numPreload frames from the new curfile onward

Definition at line 229 of file FileSystemDataSource.cc.

Referenced by advance().

const std::string & nextName (  )  [virtual]

Returns a descriptive name of the next data segment for user feedback (e.g. a filename for logged data).

Just use your class name if you don't have a useful name for individual samples.

Implements DataSource.

Definition at line 32 of file FileSystemDataSource.cc.

unsigned int nextTimestamp (  )  [virtual]

Returns the simulator time of the next data segment.

Should be in the future if nothing new since last data segment, otherwise should be the timestamp of the most recent data segment (older segments are skipped), return -1U if there is no more data

Implements DataSource.

Definition at line 28 of file FileSystemDataSource.cc.

Referenced by plistValueChanged(), resetPoller(), and DataThread::run().

FileSystemDataSource& operator= ( const FileSystemDataSource  )  [private]

don't call

void plistValueChanged ( const plist::PrimitiveBase pl  )  [virtual]

Implements PrimitiveListener.

Definition at line 294 of file FileSystemDataSource.cc.

Referenced by loadXML(), and registerSource().

void plistValueTouched ( const plist::PrimitiveBase pl  )  [virtual]

Reimplemented from PrimitiveListener.

Definition at line 288 of file FileSystemDataSource.cc.

void preprepare ( const FileSystemDataSource::files_t::iterator &  fi  )  [protected, virtual]

calls prepare on fi, dumping diagnostic info if verbose is high enough

Definition at line 556 of file FileSystemDataSource.cc.

Referenced by advance(), loadFileList(), nextFrame(), and setFrame().

void registerSource (  )  [virtual]

User hook, called when the data source should claim which outputs it provides feedback (providingOuput()).

Does not indicate the data source should start sending updates yet — wait for enteringRealtime() or advance() to be called

Reimplemented from DataSource.

Reimplemented in FileSystemSensorSource.

Definition at line 85 of file FileSystemDataSource.cc.

void resetPoller (  )  [protected, virtual]
virtual bool sendData (  )  [protected, pure virtual]

Gives the data referenced by curfile to the simulator.

Implemented in FileSystemImageSource, and FileSystemSensorSource.

Referenced by advance().

void setFrame ( unsigned int  f,
unsigned int  numPreload = 2 
) [virtual]

sets the next frame to be sent (e.g. pass 0 to reset to the first frame)

prepares (pre-loads) numPreload frames from the new curfile onward

Definition at line 210 of file FileSystemDataSource.cc.

void setLoopTime ( double  t  )  [virtual]

adds time to the final frame to increase total sequence time to t milliseconds

Definition at line 265 of file FileSystemDataSource.cc.

Referenced by LoggedDataDriver::plistValueChanged().

virtual bool usingIndexFile (  )  const [virtual]

returns indexed

Definition at line 57 of file FileSystemDataSource.h.

Referenced by plistValueChanged().


Member Data Documentation

double actualLoopTime [protected]

time it would take to run through all of the loaded frames

Definition at line 178 of file FileSystemDataSource.h.

Referenced by clearFiles(), getLoopTime(), loadFileList(), and setLoopTime().

files_t::iterator curfile [protected]

an iterator referencing files -- indicates next file to send

Definition at line 171 of file FileSystemDataSource.h.

Referenced by advance(), clearFiles(), loadFileList(), nextFrame(), nextName(), nextTimestamp(), plistValueChanged(), preprepare(), and setFrame().

a regular expression (POSIX.2 extended format) to select which files to load from path, if path is a directory or index file

Definition at line 80 of file FileSystemDataSource.h.

Referenced by deregisterSource(), FileSystemDataSource(), loadFileList(), loadFileListFromDirectory(), loadFileListFromIndex(), plistValueChanged(), and registerSource().

The sensors updates per second which should be loaded. Note this is limited by the global Sensors.Framerate setting.

Definition at line 86 of file FileSystemDataSource.h.

Referenced by deregisterSource(), FileSystemDataSource(), loadFileListFromDirectory(), loadFileListFromIndex(), loadSingleFile(), plistValueChanged(), and registerSource().

unsigned int freezeTime [protected]

time at which doFreeze was called

Definition at line 175 of file FileSystemDataSource.h.

Referenced by doFreeze(), doUnfreeze(), loadFileList(), and setFrame().

bool indexed [protected]

true if the file list was specified by an index file

Definition at line 174 of file FileSystemDataSource.h.

Referenced by loadFileListFromDirectory(), loadFileListFromIndex(), loadSingleFile(), and usingIndexFile().

double initialDelay [protected]

milliseconds to wait before sending first frame

Definition at line 172 of file FileSystemDataSource.h.

Referenced by calcLoopTime(), clearFiles(), loadFileListFromIndex(), nextFrame(), and preprepare().

controls whether to restart curfile at the beginning of files when it reaches the end

Definition at line 83 of file FileSystemDataSource.h.

Referenced by deregisterSource(), FileSystemDataSource(), nextFrame(), plistValueChanged(), registerSource(), and setFrame().

const unsigned int MAX_LOAD = 4000 [static, protected]

maximum number of data elements to try to keep 'active'. If there's more than this in files, we'll only load one at time, and immediately release it afterward

Definition at line 97 of file FileSystemDataSource.h.

Referenced by nextFrame(), and setFrame().

double naturalLoopTime [protected]

time it would take to run through all of the frames as set immediately after load (i.e. no setLoopTime())

Definition at line 179 of file FileSystemDataSource.h.

Referenced by clearFiles(), getLoopTime(), and loadFileList().

LoggedDataDriver& parent [protected]

device driver this is a member of

Definition at line 177 of file FileSystemDataSource.h.

Referenced by getUsedPath(), nextFrame(), and plistValueChanged().

plist::Primitive<std::string> path

The directory, data file, or index file from which to load via call to loadFileListFromDirectory(), loadSingleFile(), or loadFileListFromIndex().

A single file can be either a single data file (e.g. sensor or camera image), or an index file as output by VisionGUI, or in the format 'filename <tab> time', where 'filename' is an absolute path or relative to the directory containing the index file, and 'time' is in milliseconds, relative to the time at which the index file is loaded.
In the future, this could also be network addresses for teleoperation and remote processing.

Definition at line 77 of file FileSystemDataSource.h.

Referenced by deregisterSource(), FileSystemDataSource(), getUsedPath(), loadXML(), LoggedDataDriver::LoggedDataDriver(), LoggedDataDriver::plistValueChanged(), plistValueChanged(), plistValueTouched(), and registerSource().

bool registered [protected]

synced to registerSource/deregisterSource so we don't start the thread if not actually in use

Definition at line 181 of file FileSystemDataSource.h.

Referenced by deregisterSource(), isRegistered(), registerSource(), and resetPoller().

const plist::Primitive<double>* timeScale [protected]

simulator's timeScale parameter, so we can register for updates when it changes

Definition at line 176 of file FileSystemDataSource.h.

Referenced by deregisterSource(), enteringRealtime(), leavingRealtime(), plistValueChanged(), resetPoller(), and ~FileSystemDataSource().

Controls how much feedback to give on the console regarding progress.

0 - none
1 - report when messages are dropped
2 - also report when a message is sent
3 - also report when loop or end-of-log occurs
4 - also report when each message is preloaded

Definition at line 94 of file FileSystemDataSource.h.

Referenced by advance(), FileSystemDataSource(), nextFrame(), and preprepare().


The documentation for this class was generated from the following files:

Tekkotsu Hardware Abstraction Layer 5.1CVS
Generated Mon May 9 05:01:41 2016 by Doxygen 1.6.3