K4U Documentation


Joints

This category contains functions responsible for retrieving the joint specific data of the human bodies detected by the Kinect device, checking the confidence level the device has in the data it is providing and some functions that return derived data such as smoothed orientations and delta positions. The majority of the functions in this category require that a joint type and a body number be supplied as input.

The following shows each accessible Joint found in EKinectJoints and where it lies on the human body.

Hands

The Hands category of the Kinect Interface Component contains functions responsible for interacting with the hand state data of the human bodies detected by the Kinect device. The features of this category require a grounding in the possible Kinect hand states.

Body Tracking

The Body Tracking category contains functions responsible for retrieving the human bodies detected by the Kinect device and their current tracking state. It also provides functions to retrieve tracked bodies that best meet certain conditions.

Coordinate Mapping

The Coordinate Mapping functions are utilities which allow for easy conversion of 3D vectors into positions on the frame of two of the available camera frames as well as the appropriate UV coordinates of a supplied pixel coordinates. The functions that support these features are:

Leaning

This category contains functions that expose the amount a player is leaning from a vertical position at their hips. These values are the amount that the person is leaning forward/backward and left/right.

Smoothing

The Smoothing category of the Kinect Interface Component contains functions responsible for managing how smoothing of joint orientations is calculated when there is a spike in the orientation delta from one tick to the next detected.

The results of smoothing can be accessed through the Get Smoothed Joint Orientation function on the Kinect Interface Component and the Get Converted Joint Orientation on the Kinect Anim Instance class. The Kinect Anim Instance and Kinect Poseable Mesh Component in the Avateering API contain flags to signal whether smoothing should be used in their cases.

Device Checking

Audio

Get Distance Between Joints

Description:

A convenience function that calculates current distance (centimetres) between two joints on a single body. The value is found by calculating the difference between the absolute positions of the supplied joints.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Start Joint [EKinectJoints]: The joint input used as the start of the distance calculation.
  • End Joint [EKinectJoints]: The joint input used to mark the end of the distance calculation.
  • Body Number [EBodyNumber]: The body to which both joints belong.

Outputs:

  • Return value [Float]: The distance in Unreal Units (centimetres) between the positions of the two supplied joints.

Get Joint Absolute Position

Description:

Provides a vector position of the joint supplied relative to the kinect device in Unreal units (centimetres).

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The target joint of this operation.
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return Value [Vector]: An absolute position FVector for the specified joint, in Unreal units (centimetres).

Get Joint Confidence

Description:

Provides current confidence the Kinect has in the values it is supplies for the given joint. The confidence is represented being either tracked, inferred or not tracked.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The joint whose confidence value is sought.
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return value [ETrackingConfidence]: The current confidence value of the Kinect device in its tracking of the supplied joint.

Get Joint Delta Position

Description:

Provides the change in the position of a body joint tracked by the Kinect device. The value is provided Unreal coordinates with change measured in meters per second.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The joint whose delta position is being sought.
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return Value [Vector] : The current position delta in meters per second.

Get Joint Orientation

Description

Provides access to the current orientation of the a body joint tracked by the Kinect device. This is provided as a Rotator converted to Unreal world co-ordinates.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The target joint whose orientation is being sought
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return Value [Rotator]: The current orientation of the joint provided as input.

Get Smoothed Joint Orientation

Description:

Provides access to the current orientation of the a body joint tracked by the Kinect device with smoothing applied to it. This provided as a Rotator and converted to Unreal world co-ordinates. How smoothing values are reached can be controlled.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The target joint whose smoothed orientation is being sought.
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return Value [Rotator]: The current smoothed orientation of the joint provided as input.

Get Joint Relative Position

Description:

This convenience function provides the relative position (or direction in space) of the one joint tracked by the Kinect device to another. This can be used to find the position between different bodies tracked by the device as the joints are retrieved from target bodies supplied to the function.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Start Joint [EKinectJoints]: The joint whose current position provides the base for the relative vector calculation.
  • Start Joint Body Number [EBodyNumber]: The body which the start joint belongs to.
  • End Joint [EKinectJoints]: The joint whose current position provides the end point for the relative vector calculation.
  • End Joint Body Number [EBodyNumber]: The body which the end joint belongs to.

Outputs:

  • Return Value [Vector]: The relative position of the end joint to the start joint, with units in Unreal Coordinates (centimetres).

Get Pitch Adjusted Joint Position

Description:

This function modifies the position of the target joint to take into account the current pitch angle of the Kinect device in relation to horizontal. This allows for more consistent values to be returned irrespective of the angle of the device. The current value returned by Get Kinect Pitch is used to calculate this vector.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Joint Type [EKinectJoints]: The joint whose adjusted position is sought.
  • Body Number [EBodyNumber]: The body to which the target joint belongs.

Outputs:

  • Return Value [Vector] : The current position of the joint adjusted for the current pitch of the Kinect device.

Get Microphone Volume Intensity

Description:

Provides the current volume intensity of the audio being received by the microphone on the Kinect device relative to the background noise. Will provide output within a set range [0 - 1], with zero for no detectable noise above the ambient background, and 1 for the loudest noise detectable.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.

Outputs:

  • Return Value [Float]: The volume intensity of the audio from the Kinect device microphone.

Get Audio Beam Rotation In Degrees

The direction relative to the Kinect device that the microphone is actively “listening” towards. The degrees value is within a set range [-50 - +50 degrees]. The value will be clamped at the upper and lower limits in situations where audio is detected beyond those angles. The Kinect will change Beam Rotation towards the loudest noise in the room automatically by default. This can be changed with the Set Audio Beam Mode function.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.

Outputs:

  • Return Value [Float]: The degrees of rotation from the Kinect center line from which the current loudest noise has been detected.

Set Audio Beam Mode

Description:

This allows for control of the audio beam angle determination. Automatic will tell the Kinect to focus its audio beam to listen to the loudest audio source within its listening cone (+/- 50 degrees from the centre line of the device). Manual will stop the Kinect from updating this angle and remain at the last point until the mode is changed to automatic or it is updated using the Set Audio Beam Rotation In Degrees function.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Beam Mode [EAudioBeamMode]: The Audio Beam mode to be set.

Set Audio Beam Rotation In Degrees

Description:

Allows you to set the direction relative to the Kinect device that the microphone is actively “listening” towards. The degrees value is restrict to values between -50 and +50 degrees. Inputs outside the upper and lower limits will be clamped to them. The Kinect Audio Beam mode if set to automatic will override this value immediately. If set to manual the value will remain.

Inputs:

  • Target [KinectInterfaceComponent]: The Unreal Engine/Kinect interface object.
  • Angle [Float]: The degrees of rotation from the Kinect center line from which the Audio beam will focus on.