Work with Visualize Streaming

Overview

The Visualize SDK file format VSFX has built-in streaming support. This means that if a VSFX file can be provided as a set of binary arrays, Visualize SDK can consistently process these arrays and generate a Visualize database using content from these arrays. Also, by providing "enough" arrays, even an incomplete database can be rendered. Note that Visualize SDK does not keep parsed data in memory, so an incomplete database keeps in memory only the unparsed part of source data. However, due to some limitations, a regular VSFX file should be preprocessed before it can be efficiently streamed. This article describes the preprocessing phase and the process of collecting database data.

VSFX Preprocessing

Streaming Compatibility

A VSFX file can be represented as a set of sections. All sections can be divided into two groups: object sections (which contain Visualize object data: entities, geometry, devices, etc.) and service sections (which describe how object sections should be parsed).

Service sections depend on object sections, so when you generate a VSFX file, service sections are placed after object sections. Streaming such a file is completely ineffective: only after the whole content of the file is streamed can Visualize SDK read it. Such a file is called streaming-incompatible.

OdTvDatabaseReceiver has static methods that can check whether a VSFX file is streaming-compatible and has methods that can convert a streaming-incompatible VSFX file into a streaming-compatible file:

static bool isStreamingCompatible( const OdString& fileName, OdTvResult* pRes = NULL );
static bool isStreamingCompatible( OdStreamBuf* pBuffer, OdTvResult* pRes = NULL );
static OdTvResult makeStreamingCompatible( const OdString& input, const OdString& output );
static OdTvResult makeStreamingCompatible( OdStreamBuf* pInput, OdStreamBuf* pOutput );

Conversion to a streaming-compatible file is a fast enough process since Visualize SDK only swaps VSFX file sections without parsing their content.

Also, general Visualize SDK VSFX file-read methods are equally effective in both streaming-compatible and streaming-incompatible cases.

The following image shows the difference between section positions in streaming-compatible and streaming-incompatible cases:

Streaming-friendly Files

The main goal of VSFX streaming is to render a Visualize database when only part of a VSFX file is parsed. However, along with renderable objects (entities, geometry, inserts), object sections also contain objects that are required for rendering: devices, views, linetypes, etc. Therefore, rendering an incomplete database that has not received all required objects may be impossible (no devices or views are parsed) or may give results that significantly differ from rendering a complete database (wrong colors, linetypes, etc.).

In a streaming-friendly VSFX file, object sections are composed in such a way that all objects required for rendering are placed before renderable objects. These files can be rendered correctly after all required objects are parsed.

In a non-streaming-friendly VSFX file, an incomplete database can also be rendered, but in this case a client application needs to make sure that the database already parsed all necessary objects. For example, it may check the device and view iterators, linetypes iterators, etc.

General VSFX file-write methods always generate streaming-friendly content. However, unloading database partial objects (such as low memory import) may generate non-streaming-friendly files.

The isStreamingFriendlySource() method checks whether data parsing is streaming-friendly, and if it returns true, the isDatabaseRenderable() method notifies that the database is safe to render.

Suppose an object section contains four types of data: devices, layers, entities and geometry. Device and layer data is required for rendering. So if some layers have a color, entities on layers assigned "Color by Layer" are rendered incorrectly if these layers are not loaded before such entities are rendered (since in this case the default layer is used). The remaining data – entities and geometry – is renderable data. When the renderable objects are loaded, the rendering scene receives more details, but if some of these objects are missing they are not drawn and it does not affect previously drawn objects.

The following image shows the difference between streaming-friendly and non-streaming-friendly cases for the described example:

Collecting the Database in a Client Application

To collect the database on the client side:

1. Use the OdTvFactory instance to create an instance of OdTvDatabaseReceiver:

   OdTvFactoryId factId = odTvGetFactory();
   OdTvDatabaseReceiverPtr pDbReceiver = factId.createDatabaseReceiver();

2. The receiver should process each input data array:

   /*Input: 
       const OdUint8* pData; //part of .VSFX file
       OdUInt32 nDataSize; //length of pData array
   */
   bool bDone = pDbReceiver->doReceive( pData, nDataSize);

3. The doReceive method returns true if the database is completely received. In this case, the data receiving process can be stopped. Otherwise, the application should decide whether an incomplete database is ready for rendering. First, check the receiving status represented by the DatabaseReceivingState enumeration value:

   OdTvDatabaseReceiver::DatabaseReceivingState state = pDbReceiver->state();

   The state can be one of the following:
   • OdTvDatabaseReceiver::kReceiver_Nothing — Nothing is received; the database is empty.
   • OdTvDatabaseReceiver::kReceiver_AwaitingServiceData — Data parsing is started, but currently even service sections are not complete so the database is still empty.
   • OdTvDatabaseReceiver::kReceiver_AwaitingObjectsData — Data parsing is started and all service sections are received; starting from this state, an application can access the database and try to render its content.
   • OdTvDatabaseReceiver::kReceiver_Complete — Database is completely received and ready for further processing without limitations.

   If the state is lower than OdTvDatabaseReceiver::kReceiver_AwaitingObjectsData, the application should continue receiving data without database processing; if the state is higher than OdTvDatabaseReceiver::kReceiver_AwaitingObjectsData, the database is completely received.

   If the state is OdTvDatabaseReceiver::kReceiver_AwaitingObjectsData (since in the majority of cases the biggest part of a VSFX file contains the object sections), the application should check whether the VSFX source is streaming-friendly, and if so, the application can use the isDatabaseRenderable() method to check whether it can render the database:

   if( state == OdTvDatabaseReceiver::kReceiver_AwaitingObjectsData )
   {
       if( pDbReceiver->isStreamingFriendlySource() ) return pDbReceiver->isDatabaseRenderable();
   }

4. If the VSFX source file is not streaming-friendly, the application should manually check whether it can open all objects that are necessary for rendering. For example, the following code checks whether a database contains the first device and at least one view in it:

   OdTvDatabaseId dbID = pDbReceiver->database();
   OdTvDatabasePtr pDb = dbID.openObject( OdTv::kForRead );
   if( pDb.isNull() ) return false;
   OdTvDevicesIteratorPtr pDevIt = pDb->getDevicesIterator();
   if( pDevIt.isNull() ) return false;
   OdTvGsDeviceId devId = pDevIt->getDevice();
   if( devId.isNull() ) return false;
   OdTvGsDevicePtr pDevice = devId.openObject( OdTv::kForRead );
   bool anyView = false;
   OdTvResult tvRes = tvOk;
   int nViews = pDevice->numViews( &tvRes );
   if( tvRes != tvOk || nViews == 0 ) return false;
   for( int i = 0; i < nViews; ++i )
   {
     if( !pDevice->viewAt( i ).isNull() ) anyView = true;
   }
   return anyView;

Example

The OdVisualizeFirstApp sample application has a streaming tool implemented. Since it is not a network application, it emulates streaming by reading binary data chunks from a VSFX file. It also has a streaming-compatibility check and conversion for input VSFX files, which should usually be implemented on the server side.

To enable the streaming tool in the OdVisualizeFirstApp application, the OD_VISUALIZE_APP_ENABLE_STREAMING define needs to be uncommented in CommonApplications/Visualize/Examples/win/OdVisualizeFirstApp/resource.h. Example code location: OdVisualizeAppStreaming.h and OdVisualizeAppStreaming.cpp.