| threaddb 2.0 A file mapped memory container extension

Functions
threaddbC.h File Reference

List of interface methods to the database. More...

`#include "threaddbTypes.h"`

Go to the source code of this file.

## Functions
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Create (void **pThreadDB_p, size_t PackageSize_p, size_t PackageCacheLimit_p, const char *pPackagesFolder_p)
| Interface for creating a new database object. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Import (void **pThreadDB_p, const char *pIndexFileUTF8_p, size_t PackageCacheLimit_p)
| Interface for creating a new database object based on a database index file. More...
DLLEXPORT_ void | ThreadDB_Destroy (void *pThreadDB_p)
| Interface to destroy an existing database object. More...
const DLLEXPORT_ char * | ThreadDB_GetVersionInfo (void *pThreadDB_p)
| Interface for retrieving the package version info. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Save (const char *pIndexFileUTF8_p, void *pThreadDB_p)
| Interface to save the stored data. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_NewThread (const char **pFileName_p, const char *pFolder_p, size_t MaxFileSize_p, void *pThreadDB_p)
| Interface to create and register a new worker thread. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_NewPackage (uint64_t *pPackageID_p, void *pThreadDB_p)
| Registers a new data item package. More...
DLLEXPORT_ size_t | ThreadDB_GetThreadCount (void *pThreadDB_p)
| Returns the number of currently running threads. More...
DLLEXPORT_ size_t | ThreadDB_GetPackageCount (void *pThreadDB_p)
| Returns the number of currently registered packages. More...
DLLEXPORT_ size_t | ThreadDB_GetFileCount (void *pThreadDB_p)
| Returns the number of currently registered temporary database files. More...
DLLEXPORT_ size_t | ThreadDB_GetPackageSize (void *pThreadDB_p)
| Returns the package buffer size. More...
const DLLEXPORT_ char * | ThreadDB_GetDatabaseFileName (size_t FileIndex_p, void *pThreadDB_p)
| Returns the filename of a currently registered temporary database files. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RelocateFileTo (size_t FileID_p, const char *pFilePathUTF8_p, threadDB_RelocationType RelocationType_p, void *pThreadDB_p)
| Renames and copies/moves a temporary database file to a different location or disk. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Store (uint64_t Package_p, size_t Size_p, const char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
| Stores database package items in the specified package. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_SynchronizePackage (uint64_t Package_p, void *pThreadDB_p)
| Synchronizes the internal package buffers with the temporary database files. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Synchronize (void *pThreadDB_p)
| Synchronizes the internal package buffers with the temporary database files. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Open (threadDB_ReadInfo **pReadInfo_p, uint64_t Package_p, void *pThreadDB_p)
| This operation opens a package for stream reading. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_OpenItem (threadDB_ReadInfo **pReadInfo_p, const threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
| This operation opens a package for stream reading at the given item cursor. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Close (threadDB_ReadInfo **pReadInfo_p, void *pThreadDB_p)
| This operation closes a package after stream reading has finished. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RecoverContinous (size_t *pReadBytes_p, size_t Size_p, char pData_p[], threadDB_ReadInfo *pReadInfo_p, void *pThreadDB_p)
| Stream reading operation through the data items stored in a package. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RecoverRandom (uint32_t Size_p, char pData_p[], const threadDB_ItemInfo *pItemHandle_p, uint64_t Package_p, void *pThreadDB_p)
| Random access operation to the data items stored in a package. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RecoverRandomContinous (size_t Size_p, char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
| Random access operation to the data items stored in a package. More...
DLLEXPORT_ bool | ThreadDB_PackageEnd (threadDB_ReadInfo *pReadInfo_p, void *pThreadDB_p)
| Stream reading operation through the data items stored in a package. More...
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Replace (size_t Size_p, const char pData_p[], const threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
| Replace operation to modify the contents of a stored data item. More...
const DLLEXPORT_ char * | ThreadDB_GetErrorMessage (void *pThreadDB_p)
| Information on error state. More...

## Detailed Description

List of interface methods to the database.

This module contains the list of available methods provided. The interface is provided as a C header which allows to integrate the threadDB implementation as dynamic library (dll).

## Function Documentation

## ◆ ThreadDB_Close()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Close | ( | threadDB_ReadInfo ** | pReadInfo_p,
|  | void * | pThreadDB_p
| ) |  |

This operation closes a package after stream reading has finished.

This operation is required to close a package and remove the `pReadInfo_p` after stream reading has finished.

Parameters

  pReadInfo_p | Read handle created during the ThreadDB_Open operation.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Create()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Create | ( | void ** | pThreadDB_p,
|  | size_t | PackageSize_p,
|  | size_t | PackageCacheLimit_p,
|  | const char * | pPackagesFolder_p
| ) |  |

Interface for creating a new database object.

Creates a new database object and returns the related handle. The operation initializes the internal data structures and starts the thread for package buffer flushing. The `PackageSize_p` is the size of the memory block allocated for interim buffering of the stored data items. The `PackageCacheLimit_p` defines an upper limit of parallel held item buffers of size `PackageSize_p`. If the number of packages exceeds this limit the flush threads starts to withdraw package buffers from memory. This treatment requires additional thread synchronization taking place and could reduce the throughput significantly. The major part of consumed memory at a time is given by the number of packages times the package buffer size. Prior storing items in the database, it is important to create at least one worker thread using ThreadDB_NewThread.

Parameters

  pThreadDB_p | Handle to the created database object.
  PackageSize_p | Record size for buffering the data items.
  PackageCacheLimit_p | Maximum number of package buffers held in memory at one point of time.
  pPackagesFolder_p | Directory path to store the package header information

Returns
  Error code.

## ◆ ThreadDB_Destroy()

DLLEXPORT_ void ThreadDB_Destroy | ( | void * | pThreadDB_p | ) |

Interface to destroy an existing database object.

Destroys an existing database object, removes the related temporary database files and stops all running threads.

Parameters

  pThreadDB_p | Handle to the database object.

## ◆ ThreadDB_GetDatabaseFileName()

const DLLEXPORT_ char* ThreadDB_GetDatabaseFileName | ( | size_t | FileIndex_p,
|  | void * | pThreadDB_p
| ) |  |

Returns the filename of a currently registered temporary database files.

Returns the filename of the registered temporary database files with index `FileIndex_p`. During execution of this command registering of new packages using ThreadDB_GetDatabaseFileName is synchronized.

Parameters

  FileIndex_p | Index of the temporary database file.
  pThreadDB_p | Handle to the database object.

Returns
  Filename of the temporary database file.

## ◆ ThreadDB_GetErrorMessage()

const DLLEXPORT_ char* ThreadDB_GetErrorMessage | ( | void * | pThreadDB_p | ) |

Information on error state.

This function returns information on current error state of the database.

Parameters

  pThreadDB_p | Handle to the database object. return Information on the occured error.

## ◆ ThreadDB_GetFileCount()

DLLEXPORT_ size_t ThreadDB_GetFileCount | ( | void * | pThreadDB_p | ) |

Returns the number of currently registered temporary database files.

Returns the number of currently registered temporary database files.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Number of registered database files.

## ◆ ThreadDB_GetPackageCount()

DLLEXPORT_ size_t ThreadDB_GetPackageCount | ( | void * | pThreadDB_p | ) |

Returns the number of currently registered packages.

Returns the number of currently registered packages.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Number of registered packages.

## ◆ ThreadDB_GetPackageSize()

DLLEXPORT_ size_t ThreadDB_GetPackageSize | ( | void * | pThreadDB_p | ) |

Returns the package buffer size.

Returns the size used to allocate internal package buffer blocks.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Size of the internal package buffers.

## ◆ ThreadDB_GetThreadCount()

DLLEXPORT_ size_t ThreadDB_GetThreadCount | ( | void * | pThreadDB_p | ) |

Returns the number of currently running threads.

Returns the number of currently running threads.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Number of executing threads.

## ◆ ThreadDB_GetVersionInfo()

const DLLEXPORT_ char* ThreadDB_GetVersionInfo | ( | void * | pThreadDB_p | ) |

Interface for retrieving the package version info.

Provides the current version of the ThreadDB project.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Version information string.

## ◆ ThreadDB_Import()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Import | ( | void ** | pThreadDB_p,
|  | const char * | pIndexFileUTF8_p,
|  | size_t | PackageCacheLimit_p
| ) |  |

Interface for creating a new database object based on a database index file.

Creates a new database object based on a previously saved database. The operation initializes the internal data structures, starts the thread for package buffer flushing and builds up the internal package table. The `PackageSize_p` is the size of the memory block allocated for interim buffering of the stored package items. The `pIndexFileUTF8_p` needs to specify the full path to the location of the database index file. The `pIndexFileUTF8_p` path characters have to be provided in UTF8 representation. Threads registered on an imported database will create new temporary database files and the imported database files are kept mainly untouched.

Parameters

  pThreadDB_p | Handle to the created database object.
  pIndexFileUTF8_p | Record size for buffering the data items.
  PackageCacheLimit_p | Maximum number of package buffers held in memory at one point of time.

Returns
  Error code.

## ◆ ThreadDB_NewPackage()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewPackage | ( | uint64_t * | pPackageID_p,
|  | void * | pThreadDB_p
| ) |  |

Registers a new data item package.

Creates a new package to receive the stored data items. Each package definition is held in memory and consumes an amount of about 180 Bytes. Storing data items within a package synchronization of threads takes place.

Parameters

  pPackageID_p | Generated package ID.
  pThreadDB_p | Handle to the database object.

Returns
  Unique package id used for the store operation.

## ◆ ThreadDB_NewThread()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewThread | ( | const char ** | pFileName_p,
|  | const char * | pFolder_p,
|  | size_t | MaxFileSize_p,
|  | void * | pThreadDB_p
| ) |  |

Interface to create and register a new worker thread.

Creates a new worker thread receiving and managing the stored item data. Each worker thread gets a temporary database file assigned to. It is possible to define multiple worker threads and define temporary folders on different discs. The created files have the format threaddb.XXXXXX with XXXXX reflecting an unique identifying number. A basic prinicple of storing data itmes is, that the data is directed to the first available thread registered. This leads to a priority to which thread/temporary file the data is written. It is therefore sensible to specify folders on discs with faster write speed (e.g. SSD discs) first.

Parameters

  pFileName_p | Full path to the created temporary database file in ASCII character representation.
  pFolder_p | Path to the temporary database file folder in ASCII character representation.
  MaxFileSize_p | Maximum size in bytes a database file is allowed to grow. Exceeding this limit a new temporary file will be created.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Open()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Open | ( | threadDB_ReadInfo ** | pReadInfo_p,
|  | uint64_t | Package_p,
|  | void * | pThreadDB_p
| ) |  |

This operation opens a package for stream reading.

This operation opens a package for stream reading the package contents. Stream reading is sensible for continious reading through the stored data items of the given package.

Parameters

  pReadInfo_p | Generated ReadInfo handle.
  Package_p | index of the package whose contents should be recovered.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_OpenItem()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_OpenItem | ( | threadDB_ReadInfo ** | pReadInfo_p,
|  | const threadDB_ItemInfo * | pItemHandle_p,
|  | void * | pThreadDB_p
| ) |  |

This operation opens a package for stream reading at the given item cursor.

This operation opens a package for stream reading at the given stored item position.

Parameters

  pReadInfo_p | Generated ReadInfo handle.
  pItemHandle_p | handle to the stored item.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_PackageEnd()

DLLEXPORT_ bool ThreadDB_PackageEnd | ( | threadDB_ReadInfo * | pReadInfo_p,
|  | void * | pThreadDB_p
| ) |  |

Stream reading operation through the data items stored in a package.

This function provides information if the package end has been reached.

Parameters

  pReadInfo_p | read handle created during the ThreadDB_Open operation.
  pThreadDB_p | Handle to the database object.

Returns
  End of package flag.

## ◆ ThreadDB_RecoverContinous()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverContinous | ( | size_t * | pReadBytes_p,
|  | size_t | Size_p,
|  | char | pData_p[],
|  | threadDB_ReadInfo * | pReadInfo_p,
|  | void * | pThreadDB_p
| ) |  |

Stream reading operation through the data items stored in a package.

This function allows to continiously read through the stored data items. The calling process has to preallocate the `pData_p` transfer buffer and provide the size of data to be read through the `Size_p` parameter. It is possible to read amongst several data items at once.

Parameters

  pReadBytes_p | Number of read data bytes.
  Size_p | number of bytes to be read from the selected package.
  pData_p | preallocated transfer buffer, requires to be preallocated and embrace at least Size_p bytes.
  pReadInfo_p | read handle created during the ThreadDB_Open operation.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_RecoverRandom()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandom | ( | uint32_t | Size_p,
|  | char | pData_p[],
|  | const threadDB_ItemInfo * | pItemHandle_p,
|  | uint64_t | Package_p,
|  | void * | pThreadDB_p
| ) |  |

Random access operation to the data items stored in a package.

This function allows to read stored data items individually. The calling process has to preallocate the `pData_p` transfer buffer and provide the size of data to be read through the `Size_p` parameter. This function reads only the selected item. Since this method reads also from internal buffers, synchronization is not required. Since this operation requires to individually seek for the entry on disk, streaming throughput es in general less in comparison to continous reading.

Parameters

  Size_p | number of bytes to be read from the selected package.
  pData_p | preallocated transfer buffer, requires to embrace at least Size_p bytes.
  pItemHandle_p | handle to the stored item.
  Package_p | package identifier the selected item belongs to.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_RecoverRandomContinous()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandomContinous | ( | size_t | Size_p,
|  | char | pData_p[],
|  | threadDB_ItemInfo * | pItemHandle_p,
|  | void * | pThreadDB_p
| ) |  |

Random access operation to the data items stored in a package.

This function allows to read stored data items individually. The calling process has to preallocate the `pData_p` transfer buffer and provide the size of data to be read through the `Size_p` parameter. It is possible to read amongst several data items at once. The parameter `pItemHandle_p` points to the selected data item. During read the handle is moved on to the following item entry. This allows to read through a series of elements continously. Since this operation requires to individually seek for the specific entry on disk, streaming throughput is in general less in comparison to continous reading.

Parameters

  Size_p | number of bytes to be read from the selected package.
  pData_p | preallocated transfer buffer, requires to embrace at least Size_p bytes.
  pItemHandle_p | handle to the stored item.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_RelocateFileTo()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RelocateFileTo | ( | size_t | FileID_p,
|  | const char * | pFilePathUTF8_p,
|  | threadDB_RelocationType | RelocationType_p,
|  | void * | pThreadDB_p
| ) |  |

Renames and copies/moves a temporary database file to a different location or disk.

The function allows to rename or copy/move a database file to a different location or disk. The command is synchronized and available also while store or read operations are in progress.

Parameters

  FileID_p | the file index of the temporary database file to be renamed/moved.
  pFilePathUTF8_p | full path to the destination to which the database file should be moved.
  RelocationType_p | type of action copy/move to take place.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Replace()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Replace | ( | size_t | Size_p,
|  | const char | pData_p[],
|  | const threadDB_ItemInfo * | pItemHandle_p,
|  | void * | pThreadDB_p
| ) |  |

Replace operation to modify the contents of a stored data item.

This function allows to modify existing data item entries in the database. It replaces the given `Size_p` of bytes by the data provided in the `pData_p` transfer buffer. The Size_p parameter has to match the size of the previously stored data item.

Parameters

  Size_p | number of bytes to be replaced.
  pData_p | preallocated transfer buffer, requires to be embrace at least Size_p bytes.
  pItemHandle_p | handle to the stored item to be replaced.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Save()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Save | ( | const char * | pIndexFileUTF8_p,
|  | void * | pThreadDB_p
| ) |  |

Interface to save the stored data.

Saves the database control structures to the given database index file. The related temporary database files are removed from the purging process executed during the database destruction phase.

Parameters

  pIndexFileUTF8_p | Full path to the database index file location in UTF8 representation.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Store()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Store | ( | uint64_t | Package_p,
|  | size_t | Size_p,
|  | const char | pData_p[],
|  | threadDB_ItemInfo * | pItemHandle_p,
|  | void * | pThreadDB_p
| ) |  |

Stores database package items in the specified package.

The function allows to append a new data item to the package @ Package_p. The item is asynchronously copied and appended to the given package. The parameter `pItemHandle_p` handle can be omitted by specifying a NULL pointer. If available, the stored index entry of the item is provided. A `pItemHandle_p` allows random access of individual package items. The `pItemHandle_p` has to be allocated and managed separately and provided by the caller. The item handles are registered in the database and updated during the ThreadDB_Synchronize operation. Please note, that the item handle is usually invalid until the ThreadDB_Synchronize operation is executed. The `Size_p` parameter specifies the size of data to be transferred. The size of the data item is NOT stored in the database and has to be managed separately. It needs to be provided during the ThreadDB_RecoverContinous and ThreadDB_RecoverRandom operations. The maximum size specified by the `Size_p` parameter. Data items exceeding the specified `PackageSize_p` need are split into multiple records and stored performing several @ ref ThreadDB_Store calls. Please note that each data item requires to be temporary duplicated to allow asynchronous storage. A special form of the ThreadDB_Store call is to flush specific packages to disk. This can be achieved by calling the function with a `Size_p` of zero. In this case only the buffer of the selected `Package_p` is synchronized while other packages stay open for writing and are not prematurely flushed. This allows to start reading of package information while writing to other packages is still in progress.

Parameters

  Package_p | package index to which the data item should be appended.
  Size_p | size of the package item to be stored.
  pData_p | pointer to the data for being stored.
  pItemHandle_p | optional parameter for random data item access.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_Synchronize()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Synchronize | ( | void * | pThreadDB_p | ) |

Synchronizes the internal package buffers with the temporary database files.

Synchronizes the package buffers by writing their contents to the database files. This operation also updates and validates the registered item handles specified during the ThreadDB_Store operation. Synchronization is required prior calling ThreadDB_RecoverContinous operation. This operation also releases the internal package buffers.

Parameters

  pThreadDB_p | Handle to the database object.

Returns
  Error code.

## ◆ ThreadDB_SynchronizePackage()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_SynchronizePackage | ( | uint64_t | Package_p,
|  | void * | pThreadDB_p
| ) |  |

Synchronizes the internal package buffers with the temporary database files.

Synchronizes the selected package by writing its contents to the database files. This operation also updates and validates the registered item handle specified during the ThreadDB_Store operation. Synchronization is required prior calling ThreadDB_RecoverContinous or operation. This operation also releases the internal package buffer.

Parameters

  Package_p | package index to which should be synchronized.
  pThreadDB_p | Handle to the database object.

Returns
  Error code.