VizKit  3.3.7
Classes | Public Member Functions | Static Public Member Functions | Private Member Functions | Private Attributes | Static Private Attributes
VizKit::VisualAudioLab Class Reference

A collection of routines dealing with processing, analyzing and interpretation of audio data. More...

#include <VisualAudioLab.h>

List of all members.

Classes

struct  BeatGuess
 Struct with values concerning beat estimation. More...
struct  RMSValue
 Struct with values concerning rms (root mean square) values. More...

Public Member Functions

void processAudioData (const VisualAudioData &visualAudioData)
 Processes the audio data chunks.
void resetData (void)
 Resets the stored values of the VisualAudioLab.
uint32 getCurrTimestampIDOfAudioData (void)
 Returns the last processed timestampID of the kVisualPluginPulseMessage.
void setCurrTimestampIDOfAudioData (const uint32 timestampID)
 Sets the last processed timestampID of the kVisualPluginPulseMessage.
void setTotalTimeOfCurrentTrack (const uint32 totalTimeInMS)
 Stores the total time of the current track.
void setStartTimeOfCurrentTrack (const uint32 startTimeInMS)
 Stores the start time of the current track.
void setStopTimeOfCurrentTrack (const uint32 stopTimeInMS)
 Stores the stop time of the current track.
bool remainingTimeOfCurrentTrackIsKnown (void)
 Answers the question whether the remaining time of the current track is known.
uint32 getRemainingTimeOfCurrentTrack (void)
 Returns the remaining time of the currently playing audio track measured in milliseconds.
uint32 getTotalTimeOfCurrentTrack (void)
 Returns the total time of the currently playing audio track measured in milliseconds.
sint16 getCurrMusicDataHistory (void)
 Returns the current music data history.
sint16 getPrevMusicDataHistory (void)
 Returns the previous music data history.
uint8 getRMSIntensity (void)
 Returns the RMS (root-mean-square) intensity.
uint8 getWaveformDataOfChannelAtIndex (const uint16 channel, const uint16 index)
 Returns the single waveform data of a specific channel at a specific index position.
const uint8 ***const getSpectrumDataArray (void)
 Returns a pointer to the spectrum data array.
const sint16 **const getWaveformDataMonoArray (void)
 Returns a pointer to the mono waveform data array.
uint8 getMonoWaveformDataAtIndex (const uint16 index)
 Returns the waveform data value of a specific index position.
uint8 getWaveformDataOfChannelAtIndexWithHistoryNum (const uint16 channel, const uint16 index, const uint16 historyNum)
 Returns the waveform data of a specific channel of a specific history at a specific index.
uint8 getMonoWaveformDataAtIndexWithHistoryNum (const uint16 index, const uint16 historyNum)
 Returns the mono waveform data of a specific history at a specific index.
uint8 getDCIntensity (void)
 Returns the DC intensity.
void checkForBeatImpulse (void)
 Checks for the beat impulse and sends a notification in case a beat impulse is detected.
uint8 getBeatMeter (void)
 Answers the question whether the current timestamp suggests a beat impulse.
void HannWindow (const int N, double TimeSamples[])
 Processes the sample data with a Hann window.
const sint8 *const getWaveformShapeArray (void)
 Returns a pointer to the waveform shape array.
uint32 getNumberOfAudioSpectrumDataEntries (void)
 Returns the number of audio spectrum data entries.
uint32 getNumberOfWaveformEntries (void)
 Returns the number of waveform data entries.
uint32 getNumberOfSpectrumEntries (void)
 Returns the number of spectrum data entries.
uint16 getMaxNumberOfWaveformShapeHistory (void)
 Returns the maximum history number.
uint16 getNumberOfDataChannels (void)
 Returns the number of data channels.
uint16 getMaxNumberOfMusicDataHistories (void)
 Returns the maximum number of histories in waveform data.
const uint32 *const getBeatHistogram (void)
 Returns a pointer to the histogram of the gaps between detected beat events.

Static Public Member Functions

static VisualAudioLabgetInstance (void)
 Returns the VisualAudioLab.
static void dispose (void)
 Disposes the VisualAudiolab.
static void setMaxNumberOfAudioDataChannels (uint16 aNumberOfMaxAudioDataChannels)
 Sets the maximum number of audio data channels (2 channels means stereo).
static void setNumberOfAudioWaveformDataEntries (uint32 aNumberOfAudioWaveformDataEntries)
 Sets the number of audio waveform data entries.
static void setNumberOfAudioSpectrumDataEntries (uint32 aNumberOfAudioSpectrumDataEntries)
 Sets the number of audio spectrum data entries.

Private Member Functions

 VisualAudioLab ()
 The constructor.
 ~VisualAudioLab ()
 The destructor.
 VisualAudioLab (const VisualAudioLab &other)
 Copy constructor.
VisualAudioLaboperator= (const VisualAudioLab &other)
 Assignment operator.
void init (void)
 Allocates and initializes the buffers of the VisualAudioLab.
void processWaveformData (uint8 numWaveformChannels, uint16 numWaveformEntries, const uint8 *const currWaveformData)
 Processes the waveform data.
void processSpectrumData (const uint8 numSpectrumChannels, const uint16 numSpectrumEntries, const uint8 *const currSpectrumData)
 Processes the spectrum data.
void incrementMusicDataHistory (void)
 Increments the current history number of music data.
void storeBeatImpulse (bool beatImpulse)
 Stores the beat impulse.
void addToWaveformShape (sint8 minWaveformPoint, sint8 maxWaveformPoint)
 Adds the minimum and maximum waveform data value to the waveform shape.
void addToBeatRecords (void)
 Adds a value to beatsRecordedInBeatAnalyseTimeFrame.
void addToBeatGapHistogram (uint16 gapInMilliseconds)
 Adds a value to beatGapHistogram.
void resetBeatGapHistogram (void)
 Resets the beatGapHistogram.
void calcBeatConfidence (void)
 Calculates the confidence of the beat estimation.
uint32 getBestBeatMSInterval (void)
 Returns the best guess for the beat interval in milliseconds.
uint8 isBeatMeter (uint32 elapsedMSSinceStartOfTrack)

Private Attributes

uint32 totalTimeOfTrack
 The total time of the currently playing audio track measured in milliseconds.
uint32 elapsedTimeOfTrack
 The accumulated play time of the current audio track measured in milliseconds.
uint32 startTimeOfTrack
 The start time of the currently playing audio track measured in milliseconds.
uint32 stopTimeOfTrack
 The stop time of the currently playing audio track measured in milliseconds.
uint32 currTimestampIDOfAudioData
 The last processed timestampID of the kVisualPluginPulseMessage.
sint8 numberOfMusicDataHistoryInt
 The number of music data history.
sint8 *** waveformData
 The waveform data array.
sint16 ** waveformDataMono
 The mono waveform data array.
uint8 *** spectrumData
 The spectrum data array.
double * amplitudeEnvelope
 The amplitude envelope array.
sint32mean
 The mean values.
double * tappingBuffer
 A special buffer for beat detection.
double ** rms
 The rms values.
double * rmsMono
 The mono rms values.
double rmsMonoAvg
 The average rms value.
uint32beatGapHistogram
 A histogram of the gaps between detected beat events.
uint32 beatTimeDeltaInMS
 The delta between beats.
double rmsMonoMax
 The maximum mono rms value.
sint16 currMusicDataHistory
 The current history number.
uint8minSpectrumLevel
 The minimum spectrum level.
uint8maxSpectrumLevel
 The maximum spectrum level.
uint8 spectrumDCLevel
 The spectrum DC level.
float avgSpectrumDCLevel
 The average spectrum DC level.
float spectrumDCLevelSum
 The sum of the spectrum DC level.
sint8minWaveformVal
 The minimum waveform data value.
sint8maxWaveformVal
 The maximum waveform data value.
sint8 minWaveformValMono
 The minimum mono waveform data value.
sint8 maxWaveformValMono
 The maximum mono waveform data value.
uint32waveformValSum
 The sum of the absolute waveform data values that are higher or lower than 128.
uint32peakWaveformValSum
 The peak values of the sum of the waveform data values.
uint32prevWaveformValSum
 The previous sum of the waveform data values.
uint32avgWaveformValSum
 The average value of the sum of the waveform data values.
uint32avgWaveformValSumCounter
 The counter for the calculation o the average value of the sum of the waveform data values.
bool isBeatImpulse
 1 if the current waveform data indicates a beat impulse, 0 if not.
RMSValue rmsBuffer [2000]
 Array containg rms mono values for the last numberOfMillisecondsOfRMSBuffer milliseconds.
uint16 currRMSValue
 The current index of array rmsBuffer.
uint8 currRecordedBeat
 The current index of array beatsRecordedInBeatAnalyseTimeFrame.
BeatGuessbeatsRecordedInBeatAnalyseTimeFrame
 Array in which the detected beats are stored with their elapsed milliseconds timestamp since start of track.

Static Private Attributes

static VisualAudioLabtheVisualAudioLab = NULL
 VisualAudioLab is a singleton class.
static uint16 maxNumberOfAudioDataChannels = 0
 The maximum number of audio data channels (2 channels means stereo).
static uint32 numberOfAudioWaveformDataEntries = 0
 The number of audio waveform data entries.
static uint32 numberOfAudioSpectrumDataEntries = 0
 The number of audio spectrum data entries.
static const uint8 beatTimeTolerance = 25
 The smallest time a beat is considered to be the same.
static const uint16 beatThreshold = 300
 The threshold after which we are free to detect a new beat.
static const uint16 beatAnalyseTimeFrame = 10000
 The time frame in which beats are recorded and analysed.
static const uint16 maximumBeatInterval = 2000
 The maximum beat interval.
static const uint16 numberOfMillisecondsOfRMSBuffer = 3000
 The number of milliseconds rms values are collected.
static const uint16 numberOfRMSBufferEntries = 2000
 The number of records in rmsBuffer.
static const uint16 maxNumberOfWaveformShapeHistory = 1400
static const uint16 maxNumberOfMusicDataHistory = 40
static const uint16 tappingBufferLength = 4096
static const uint16 maxNumberOfBeatHistogramBins = 100
static const uint16 maxNumberOfBeatsRecordedInAnalyseTimeframe = 200

Detailed Description

A collection of routines dealing with processing, analyzing and interpretation of audio data.


Constructor & Destructor Documentation

The constructor.

VisualAudioLab is a singleton class. The constructor is private. New instance of class can only be created internally.

The destructor.

VisualAudioLab is a singleton class. The destructor is private. Instance of class can only be destructed internally.

Copy constructor.

Parameters:
otherAnother VisualAudioLab.
Remarks:
Explicitely declared in private section and not implemented to enforce uniqueness of singleton pattern.

Member Function Documentation

void VisualAudioLab::addToBeatGapHistogram ( uint16  gapInMilliseconds) [private]

Adds a value to beatGapHistogram.

Parameters:
gapInMillisecondsThe gap to previous beat detection in milliseconds.
void VisualAudioLab::addToBeatRecords ( void  ) [private]

Adds a value to beatsRecordedInBeatAnalyseTimeFrame.

void VizKit::VisualAudioLab::addToWaveformShape ( sint8  minWaveformPoint,
sint8  maxWaveformPoint 
) [private]

Adds the minimum and maximum waveform data value to the waveform shape.

Parameters:
minWaveformPointThe minimum waveform data value.
maxWaveformPointThe maximum waveform data value.
void VisualAudioLab::calcBeatConfidence ( void  ) [private]

Calculates the confidence of the beat estimation.

Checks for the beat impulse and sends a notification in case a beat impulse is detected.

void VisualAudioLab::dispose ( void  ) [static]

Disposes the VisualAudiolab.

const uint32 *const VisualAudioLab::getBeatHistogram ( void  )

Returns a pointer to the histogram of the gaps between detected beat events.

Returns:
A pointer to the histogram of the gaps between detected beat events.

Answers the question whether the current timestamp suggests a beat impulse.

Returns:
1 if the current timestamp suggests a beat impulse, 0 if not.

Returns the best guess for the beat interval in milliseconds.

The best guess for the beat interval in milliseconds.

Returns the current music data history.

Returns:
The current music data history.

Returns the last processed timestampID of the kVisualPluginPulseMessage.

Returns:
The last processed timestampID of the kVisualPluginPulseMessage.

Returns the DC intensity.

The DC intensity is the intensity of the direct current.

Returns:
The DC intensity.

Returns the VisualAudioLab.

The VisualAudiolab is initialized if required. The VisualAudioLab is a singleton.

Returns the maximum number of histories in waveform data.

Returns:
The maximum number of histories in waveform data.

Returns the maximum history number.

Returns:
The maximum history number.

Returns the waveform data value of a specific index position.

Parameters:
indexThe index of the requested waveform data.
Returns:
The mono waveform data at a specific index.

Returns the mono waveform data of a specific history at a specific index.

Parameters:
indexThe index of the requested waveform data.
historyNumThe history of the requested waveform data.
Returns:
The mono waveform data of a specific history at a specific index.

Returns the number of audio spectrum data entries.

Returns:
The number of audio spectrum data entries.

Returns the number of data channels.

Returns:
The number of data channels.

Returns the number of spectrum data entries.

Returns:
The number of spectrum data entries.

Returns the number of waveform data entries.

Returns:
The number of waveform data entries.

Returns the previous music data history.

The previous music data history is the one that was current before the one that is now current.

Returns:
The previous music data history.

Returns the remaining time of the currently playing audio track measured in milliseconds.

Returns:
The remaining time of the currently playing audio track measured in milliseconds.

Returns the RMS (root-mean-square) intensity.

The RMS intensity is the computed root-mean-square value.

Returns:
The RMS intensity.
const uint8 ***const VisualAudioLab::getSpectrumDataArray ( void  )

Returns a pointer to the spectrum data array.

Returns:
A pointer to the spectrum data array.

Returns the total time of the currently playing audio track measured in milliseconds.

Returns:
The total time of the currently playing audio track measured in milliseconds.

Returns a pointer to the mono waveform data array.

The mono waveform data consists of the averaged values of the waveform data of all available channels.

Returns:
A pointer to the mono waveform data array.

Returns the single waveform data of a specific channel at a specific index position.

Parameters:
channelThe channel.
indexThe index of the waveform data.
Returns:
The single waveform data of a specific channel at a specific index position.
uint8 VisualAudioLab::getWaveformDataOfChannelAtIndexWithHistoryNum ( const uint16  channel,
const uint16  index,
const uint16  historyNum 
)

Returns the waveform data of a specific channel of a specific history at a specific index.

Parameters:
channelThe cannel of the requested waveform data.
indexThe index of the requested waveform data.
historyNumThe history of the requested waveform data.
Returns:
The waveform data of a specific channel of a specific history at a specific index.

Returns a pointer to the waveform shape array.

Returns:
A pointer to the waveform shape array.
void VisualAudioLab::HannWindow ( const int  N,
double  TimeSamples[] 
)

Processes the sample data with a Hann window.

Parameters:
NThe number of samples.
TimeSamplesThe sample data.
void VisualAudioLab::incrementMusicDataHistory ( void  ) [private]

Increments the current history number of music data.

Music data is stored for a number of histories.

void VisualAudioLab::init ( void  ) [private]

Allocates and initializes the buffers of the VisualAudioLab.

uint8 VisualAudioLab::isBeatMeter ( uint32  elapsedMSSinceStartOfTrack) [private]
Parameters:
elapsedMSSinceStartOfTrackThe number of milliseconds elapsed since start of audio track.
Returns:
1 if time is in frame of beat meter, 0 if not.
VisualAudioLab& VizKit::VisualAudioLab::operator= ( const VisualAudioLab other) [private]

Assignment operator.

Remarks:
Explicitely declared in private section and not implemented to enforce uniqueness of singleton pattern.
void VisualAudioLab::processAudioData ( const VisualAudioData visualAudioData)

Processes the audio data chunks.

With each PulseMessage iTunes delivers waveform data and spectrum data. The spectrum data is a fft (Fast Fourier Transform) of the waveform data.

Parameters:
visualAudioDataVisual audio data.
void VisualAudioLab::processSpectrumData ( const uint8  numSpectrumChannels,
const uint16  numSpectrumEntries,
const uint8 *const  currSpectrumData 
) [private]

Processes the spectrum data.

Parameters:
numSpectrumChannelsThe number of spectrum data channels (e.g. two for stereo sound)..
numSpectrumEntriesThe number of data points per channel.
currSpectrumDataA pointer to the current spectrum data.
void VisualAudioLab::processWaveformData ( uint8  numWaveformChannels,
uint16  numWaveformEntries,
const uint8 *const  currWaveformData 
) [private]

Processes the waveform data.

Parameters:
numWaveformChannelsThe number of waveform data channels (e.g. two for stereo sound).
numWaveformEntriesThe number of data points per channel.
currWaveformDataA pointer to the current waveform data.

Answers the question whether the remaining time of the current track is known.

Streamed audio has no info about remaining time of current track.

Returns:
True if the remaining time of the current track is known, false if not.
void VisualAudioLab::resetBeatGapHistogram ( void  ) [private]

Resets the beatGapHistogram.

void VisualAudioLab::resetData ( void  )

Resets the stored values of the VisualAudioLab.

The VisualAudioLab stores analyzed values of an audio track. Typically the values are resetted when a new track starts playing.

Sets the last processed timestampID of the kVisualPluginPulseMessage.

Parameters:
timestampIDThe last processed timestampID of the kVisualPluginPulseMessage.
void VisualAudioLab::setMaxNumberOfAudioDataChannels ( uint16  aNumberOfMaxAudioDataChannels) [static]

Sets the maximum number of audio data channels (2 channels means stereo).

Parameters:
aNumberOfMaxAudioDataChannelsThe maximum number of audio data channels.
void VisualAudioLab::setNumberOfAudioSpectrumDataEntries ( uint32  aNumberOfAudioSpectrumDataEntries) [static]

Sets the number of audio spectrum data entries.

Parameters:
aNumberOfAudioSpectrumDataEntriesThe number of audio spectrum data entries.
void VisualAudioLab::setNumberOfAudioWaveformDataEntries ( uint32  aNumberOfAudioWaveformDataEntries) [static]

Sets the number of audio waveform data entries.

Parameters:
aNumberOfAudioWaveformDataEntriesThe number of audio waveform data entries.
void VisualAudioLab::setStartTimeOfCurrentTrack ( const uint32  startTimeInMS)

Stores the start time of the current track.

Parameters:
startTimeInMSThe start time of the currently playing audio track measured in milliseconds.
void VisualAudioLab::setStopTimeOfCurrentTrack ( const uint32  stopTimeInMS)

Stores the stop time of the current track.

Parameters:
stopTimeInMSThe stop time of the currently playing audio track measured in milliseconds.
void VisualAudioLab::setTotalTimeOfCurrentTrack ( const uint32  totalTimeInMS)

Stores the total time of the current track.

Parameters:
totalTimeInMSThe total time of the currently playing audio track measured in milliseconds.
void VisualAudioLab::storeBeatImpulse ( bool  beatImpulse) [private]

Stores the beat impulse.

Parameters:
beatImpulseTrue if the current waveform data indicates a beat impulse, false if not.

Member Data Documentation

The amplitude envelope array.

The average spectrum DC level.

The average value of the sum of the waveform data values.

The counter for the calculation o the average value of the sum of the waveform data values.

const uint16 VisualAudioLab::beatAnalyseTimeFrame = 10000 [static, private]

The time frame in which beats are recorded and analysed.

Measured in milliseconds.

A histogram of the gaps between detected beat events.

Array in which the detected beats are stored with their elapsed milliseconds timestamp since start of track.

Ringbuffer.

const uint16 VisualAudioLab::beatThreshold = 300 [static, private]

The threshold after which we are free to detect a new beat.

Measured in milliseconds.

The delta between beats.

const uint8 VisualAudioLab::beatTimeTolerance = 25 [static, private]

The smallest time a beat is considered to be the same.

Measured in milliseconds.

The current history number.

The current index of array beatsRecordedInBeatAnalyseTimeFrame.

The current index of array rmsBuffer.

The last processed timestampID of the kVisualPluginPulseMessage.

The accumulated play time of the current audio track measured in milliseconds.

1 if the current waveform data indicates a beat impulse, 0 if not.

const uint16 VisualAudioLab::maximumBeatInterval = 2000 [static, private]

The maximum beat interval.

Measured in milliseconds.

The maximum number of audio data channels (2 channels means stereo).

The maximum spectrum level.

0-128.

The maximum waveform data value.

0-255. -128 ...127.

The maximum mono waveform data value.

The mean values.

The minimum spectrum level.

0-128.

The minimum waveform data value.

0-255. -128 ...127.

The minimum mono waveform data value.

The number of audio spectrum data entries.

The number of audio waveform data entries.

The number of milliseconds rms values are collected.

The number of music data history.

const uint16 VisualAudioLab::numberOfRMSBufferEntries = 2000 [static, private]

The number of records in rmsBuffer.

Must be greater than: Audio sample rate (44100) / kVisualNumWaveformEntries * (numberOfMillisecondsOfRMSBuffer / 1000).

The peak values of the sum of the waveform data values.

The previous sum of the waveform data values.

double** VizKit::VisualAudioLab::rms [private]

The rms values.

Array containg rms mono values for the last numberOfMillisecondsOfRMSBuffer milliseconds.

Ringbuffer. RMSValues with ((currentMSSinceStartOfTrack - timeInMillisecondsSinceStartOfTrack) > numberOfMillisecondsOfRMSBuffer) are discarded.

The mono rms values.

The average rms value.

The maximum mono rms value.

Timeframe numberOfMillisecondsOfRMSBuffer.

The spectrum data array.

The spectrum data array is 3-dimensional with data values, channels and history.

The spectrum DC level.

0-256 (?). Direct Current of FFT (mono).

The sum of the spectrum DC level.

sum of data (for calc of avg).

The start time of the currently playing audio track measured in milliseconds.

iTunes allows to set a custom start time for a track.

The stop time of the currently playing audio track measured in milliseconds.

iTunes allows to set a custom stop time for a track.

A special buffer for beat detection.

const uint16 VisualAudioLab::tappingBufferLength = 4096 [static, private]

VisualAudioLab is a singleton class.

Pointer to private instance is handled internally.

The total time of the currently playing audio track measured in milliseconds.

The waveform data array.

The waveform data array is 3-dimensional with data values, channels and history.

The mono waveform data array.

The mono waveform data array is 2-dimensional with data values, and history.

The sum of the absolute waveform data values that are higher or lower than 128.


The documentation for this class was generated from the following files:

Generated on Sun May 3 2015 20:26:31 for VizKit by doxygen 1.8.0