Readme for Kinect for Windows SDK - Beta 2 release

The Microsoft Kinect for Windows SDK provides the native and managed APIs and the tools you need to develop Kinect enabled applications for Windows. Applications that are built using the Kinect SDK can currently run on Windows 7 computers and the September 2011 build of the Windows 8 Developer Preview. Developing Kinect enabled applications is essentially the same as developing other Windows applications, except that this SDK provides support for the features of the Kinect sensor (color images, depth images, audio, skeletal data, etc.).

Contents:

1. System Requirements
2. Installation
3. Changes in this release
4. Migrating projects from Beta 1
5. Getting Help
6. Kinect Sensor Notes
7. Notes for Developers
8. C++ Header Files
9. Samples
10. Troubleshooting

1. System Requirements

Supported Operating Systems

• Windows 7, Windows 8 Developer Preview

Hardware Requirements

• 32 bit (x86) or 64 bit (x64) processor
• Dual-core 2.66-GHz or faster processor
• Dedicated USB 2.0 bus
• 2 GB RAM
• Windows 7-compatible graphics card that supports DirectX 9.0c
• A retail Kinect for Xbox 360® sensor, which includes special USB/power cabling

Software Requirements

• Microsoft® Visual Studio® 2010 Express or other Visual Studio 2010 edition
• .NET Framework 4.0
   
• For C++ Skeletal Viewer samples:
  Microsoft DirectX® SDK - June 2010 or later version
  Runtime for Microsoft DirectX® 9
   
• For Speech samples:
  Microsoft Speech Platform Runtime, version 10.2 – select 32-bit if you are running 32-bit Windows.  If you have 64-bit Windows, we suggest that you install both the 32-bit and 64-bit runtime.
  Microsoft Speech Platform - Software Development Kit, version 10.2 – select 32-bit or 64-bit according to your Windows installation
  Kinect for Windows Runtime Language Pack, version 0.9

2. Installation

To install the SDK Beta:

1.     Make sure the Kinect device is not plugged in to the USB port on your computer.
 

2.     If you installed any previous Kinect for Windows SDK, uninstall it first.  Similarly, you must remove any other drivers for the Kinect device.
 

3.     Close Visual Studio.
Note: You must close Visual Studio before installing the SDK and then restart it after installation to pick up the KINECTSDK_DIR environment variables that the SDK requires.
 

4.     From the download location for the SDK Beta package, double-click the link for your Windows installation: 32-bit or 64-bit.

5.     Once the SDK has completed installing successfully, ensure your Kinect device is plugged in to an external power source then plug the Kinect into the PC’s USB port.  The drivers will load automatically.  Your Kinect should now be working correctly.

6.     To run the samples, ensure that the DirectX and Speech prerequisites are installed.  They are listed in the previous section.

If the driver is installed correctly:

1.     You should see a blinking green LED on the Kinect sensor.
 

2.     The Kinect sensor should appear in Device Manager as the following nodes under “Microsoft Kinect”:
• Microsoft Kinect Audio Array Control
• Microsoft Kinect Camera
• Microsoft Kinect Device
 

3.     The Kinect sensor's microphone array should appear under Sound, Video and Game Controllers in Device Manager as:
• Kinect USB Audio

If you have trouble getting the sensor or SDK working, please read the “Tips and Troubleshooting” section of the SDK Documentation, in the Programming Guide section under Resources.

3. Changes in this release

Since the Beta 1 Refresh release, the following changes have been made:

• Significant improvements to skeletal tracking:
• Accuracy has been improved overall, resulting in more precise tracking.
• Skeletal Frame delivery is faster, resulting in reduced latency.
• Skeletal Tracking is now multi-threaded and can take advantage of multiple CPU cores.
• When using 2 Kinects, developers can now specify which one is used for skeletal tracking.
• API support for detecting and managing device status changes, such as device unplugged, device plugged in, power unplugged, etc.  Apps can reconnect to the Kinect device after it is plugged in, after the computer returns from suspend, etc.  See the Shape Game sample code for the best example.
• Developers using the audio within WPF no longer need to access the DMO from a separate thread.  You can create the KinectAudioSource on the UI thread and simplify your code.
• The driver, runtime, and SDK work correctly on the Windows 8 Developer Preview for desktop applications.
• The SDK can be used to build 64-bit applications.  Previously, only 32-bit applications could be built.
• NuiImageBuffer has been replaced by INuiFrameTexture, defined in MSR_NuiImageCamera.h.  It is no longer necessary to include the file NuiImageBuffer.h in your project.
• The SDK install structure and default location have changed.  The install location is in the environment variable %KINECTSDK_DIR% which defaults to C:\Program Files\Microsoft SDKs\Kinect\v1.0 Beta2
• Sample code changes:
• There are is a new C# sample: KinectAudioDemo.
• The samples have been updated.  Also the C# samples use helpers, KinectWpfViewers, that may be useful in your apps.
• The samples are now installed to Samples folder of the SDK directory, which defaults to C:\Program Files\Microsoft SDKs\Kinect\v1.0 Beta2\Samples.  Unzip the samples file to view the source code.  We recommend that you unzip them to somewhere outside of the Program Files directory.
• Driver and runtime stability and performance improvements, especially for the managed API layer.

Problems and known issues:

• When extracting the samples from the zip file, the sample source files are by default Read Only. If modifying them in Visual Studio, it will prompt to verify if it is ok to overwrite. In other text editors, to modify them, you may have to first change their file properties to read write.

• The Kinect for Windows drivers have encountered compatibility problems with certain USB host controllers, such as the Intel 5 Series/3400 Series Chipset USB host controller.  On those hubs, you may encounter a problem, such as a Not Ready status, when plugging/unplugging a Kinect.  If this happens to you, you may have to restart your app or reboot your computer.

4. Migrating projects from Beta 1

• After installing this SDK, you should recompile your apps before running them.  Existing app binaries are not guaranteed to keep working.
• To migrate existing C# projects to this SDK, you should remove and re-add the reference to Microsoft.Research.Kinect.dll.  To do that, in the Solution Explorer find the Microsoft.Research.Kinect under References, right click it to remove, then right-click on References to Add Reference.  Click on the .NET tab to find Microsoft.Research.Kinect.
• To migrate existing C++ projects to this SDK, the Inc and Lib paths need to be updated to the new locations.  Replace $(MSRKINECT)\inc and $(MSRKINECT)\lib with $(KINECTSDK_DIR)\inc and $(KINECTSDK_DIR)\lib\x86 respectively.
• To convert an existing C++ project to use the 64-bits support enabled in M3, do the same steps as above but use amd64 in the lib path instead of x86, i.e. $(KINECTSDK_DIR)\lib\amd64.  You will also have to change any other library paths in your project to point to the 64-bit versions.

5. Getting Help

The SDK documentation can be launched from the Start Menu. The Programming Guide section of the documentation has many important details for developers, including how to access the APIs from C++ and managed code projects, understanding the Natural User Interface APIs and Audio APIs, how to accomplish common tasks, etc.  The documentation also contains API reference information. This readme also contains information that may be useful.

Online developer resources for Kinect for Windows include the following:

·         Kinect for Windows SDK Beta Forums

·         Coding4Fun Kinect Showcase

·         Technical Resources on the Kinect for Windows Website

·         (coming soon) The Kinect for Windows Blog

For general Windows development questions, try the following:

• Microsoft Developer Network (http://msdn.microsoft.com): The latest development documentation to help you create applications for Windows.
• Microsoft TechNet (http://technet.microsoft.com): The latest administration documentation to help you administer the Windows operating system.

6. Kinect Sensor Notes

General Sensor Notes

1.     Important: The Microsoft Kinect drivers might not install correctly and will not work correctly if another driver for the Kinect device is installed on your PC.
 

2.     Plug the Kinect sensor's power supply into an external power source.  With USB power only, the sensor will be minimally functional and light the LED, but it must be connected to an external power source to be fully functional.
 

3.     No tools are required for calibration of audio and video.
 

4.     Your Kinect sensor should be the only device plugged into the USB hub it uses on your computer. If you have more than one Kinect sensor, make sure that you connect them to different USB hubs.
 

Protecting Your Kinect Sensor While Experimenting

Overheating

1.     The Kinect sensor is protected from overheating by a fan, which is controlled by the sensor's firmware and turns off the camera at 90 degrees Celsius (194 degrees Fahrenheit).
 

2.     There is no software interface for applications to control the fan.

Controlling the Sensors

1.     All control of the Kinect sensor is through the APIs provided in this SDK Beta.
Caution: If you use any other mechanisms to control the Kinect sensor, you risk the possibility of irreparable damage to the sensor.
 

2.     Warning: The tilt mechanism in the sensor array is not rated for frequent use. You should tilt the Kinect sensor as few times as possible, to minimize wear on the camera and to minimize tilting time. The camera motor is not designed for constant or repetitive movement, and attempts to use it that way may cause degradation of motor function.
 
In this SDK Beta, calls to NuiCameraSetElevation are limited to 1 per second and a maximum of 15 calls in any 20-second period. If the application exceeds these limits, the function imposes a short lockout period during which any further calls will return one of the following error codes:
 
ERROR_TOO_MANY_CMDS
The application tried to move the camera more than 15 times within 20 seconds.
 
ERROR_RETRY
The application must wait one second between attempts to move the camera.
 

3.     This SDK Beta does not support selective suspend for power management.

Controlling the Camera Array

1.     This SDK Beta allows applications to control camera tilt only, not pan or zoom.
 

Skeletal Tracking

1.     If moving figures are too close to the camera, skeleton tracking can produce unreliable or odd results. The optimal range is approximately 4 to 11 feet (about 1.2 to 3.5 meters).
 

2.     This SDK Beta supports the skeleton tracking pipeline, but does not include support for velocity or rotation.
 

3.     Skeleton tracking supports only two players.
 

4.     If an application specifies depth data only — and not depth with player index — the returned image data contains depth information in the low-order 12 bits of each 16-bit value.
 

5.     If you are using multiple Kinect sensors, skeleton tracking works only on the first device that you Initialize (with the RuntimeOption for skeletal tracking).  To switch the device being used to track, uninitialized the old one and initialize the new one. For an example, see the SkeletalViewer-WPF sample.
 

6.     This SDK Beta does not support the following Xbox 360 features:
• Selecting which Skeleton to track – Up to 6 users in the field of view will be reported in the depth stream; of these, 2 users will be actively tracked with skeletal data. The application developer cannot choose which of the 6 users to track actively.
• Initializing ATG simple shaders
• Implementing Picture-in-Picture (PiP)
• Gestures — although you can experiment with using depth streams for gesture-like effects
• Identity

7. Notes for Developers

General Developer Notes

1.     Visual Studio Version: Visual Studio 2010 Express (or other Visual Studio 2010 edition) is required.
 

2.     Developer Environment: For information, see the Introduction of the Programming Guide section of the documentation.
 

3.     Working with other SDKs and frameworks:
This SDK is not intended for developing prototype Xbox applications with the intent of porting those applications to the Microsoft Xbox 360 console. Microsoft recommends that developers use Xbox 360 development kits (XDK hardware and software) to create Xbox 360 applications.
 

4.     Virtual machines: You must run applications built with the Kinect for Windows SDK Beta in a native Windows environment. Kinect for Windows applications cannot run in a virtual machine, because the Microsoft Kinect drivers and this SDK Beta must be installed on the computer where the application is running.
 

Unmanaged Code

1.     Compiling and Linking: To implement a C++ application:
 
• For standard definitions, include windows.h in your source code.
 
• To use the NUI API, include MSR_NuiApi.h, which is in %KINECTSDK_DIR%\inc
 
• To use the Kinect Audio API, include MSRKinectAudio.h, which is in %KINECTSDK_DIR%\inc
 
• Link to MSRKinectNUI.lib, which is in %KINECTSDK_DIR%\lib\x86 (for 32-bit apps) or %KINECTSDK_DIR%\lib\amd64 (for 64-bit apps)
  

2.     Player Index: To use player index information, C++ applications must specify both NUI_INITIALIZE_FLAG_USES_DEPTH_AND_PLAYER_INDEX and NUI_INITIALIZE_FLAG_USES_SKELETON in the dwFlags parameter to NuiInitialize.
 

Managed Code

1.     References: Managed code applications that use this SDK Beta must reference Microsoft.Research.Kinect.dll. This assembly is in the GAC, and can be found on the .NET tab of the Add Reference dialog box.
 

2.     Namespaces: Include using directives for the following namespaces:
 
For the NUI API, add:
 
using Microsoft.Research.Kinect.Nui;
 
For the Audio API, add:
 
using Microsoft.Research.Kinect.Audio;
 

3.     Player Index: For player index information to work properly, managed code applications that specify RuntimeOptions.UseDepthAndPlayerIndex must also specify RuntimeOptions.UseSkeletalTracking.
 

8. C++ Header Files

Here is a description of the functionality exposed in the C++ header files, located in %KINECTSDK_DIR%\inc

MSR_KinectAudio.h
Defines the audio API, including the ISoundSourceLocalizer interface, which supports beamforming and source localization.

MSR_NuiApi.h
Aggregates all NUI API headers and defines basic initialization and access functions—NuiInitialize, NuiShutdown, MSR_NuiXxx, and INuiInstance:
• Enumerate devices
• Access multiple devices

MSR_NuiImageCamera.h
Defines API for the NUI image and camera services—NuiCameraXxx, NuiImageXxx:
• Adjust camera tilt
• Open streams and read image or depth frames

MSR_NuiProps.h
Defines the APIs for the NUI properties enumerator.

MSR_NuiSkeleton.h
Defines API for NUI Skeleton—NuiSkeletonXxx, NuiTransformXxx:
• Enable/disable skeleton tracking
• Get skeleton data
• Transform skeleton data for smoother rendering

9. Samples

Security Note: The sample code in this SDK Beta is designed to illustrate concepts and may not illustrate the safest coding practices. Microsoft assumes no liability for incidental or consequential damages should the sample code be used for purposes other than as intended.

The SDK Beta includes the following samples:

Skeletal Viewer—Capturing Data with the NUI API (C++ and C#)
The Kinect sensor includes two cameras: one delivers depth information and the other delivers color data. The NUI API enables applications to access and manipulate this data. The SkeletalViewer sample uses the NUI API to render data from the Kinect sensor’s cameras as images on the screen. The managed sample uses WPF to render captured images, and the native application uses DirectX.

ShapeGame—Creating a Game with Audio and Skeletal Tracking (C#)
Displays the tracked skeletons of two players together with shapes falling from the sky. Players can control the shapes by moving and speaking commands.

Audio Capture Raw (C++)
The Kinect sensor’s audio component is a four-element microphone array. The AudioCaptureRaw sample uses the Windows Audio Session API (WASAPI) to capture the raw audio stream from the Kinect sensor’s microphone array and write it to a .wav file.

MicArrayEchoCancellation—Acoustic Echo Cancellation, Beam Forming, and Source Localization (C++)
The primary way for C++ applications to access the Kinect sensor’s microphone array is through the MSRKinectAudio DirectX Media Object (DMO). The MSRKinectAudio DMO supports all standard microphone array functionality, and adds support for beamforming and source localization. The MicArrayEchoCancellation sample shows how to use the KinectAudio DMO as a DirectShow® source to access the Kinect sensor’s microphone array. This sample uses acoustic echo cancellation to record a high-quality audio stream and beamforming and source localization to determine the selected beam and the direction of the sound source.

MFAudioFilter—Media Foundation Audio Filter (C++)
Shows how to capture an audio stream from the Kinect sensor’s microphone array by using the MSRKinectAudio DMO in filter mode in a Windows Media Foundation topology.

RecordAudio—Recording an Audio Stream and Monitoring Direction (C#)
Demonstrates how to capture an audio stream from the Kinect sensor’s microphone array and monitor the currently selected beam and sound source direction.

Speech—Recognizing Voice Commands (C#)
Demonstrates how to use the Kinect sensor’s microphone array with the Microsoft.Speech API to recognize voice commands.

Kinect Audio Demo (C#)
Demonstrates how to capture an audio stream from the Kinect sensor’s microphone array and integrate it with the Microsoft Speech SDK for speech recognition. Also demonstrates visualization of sound wave and sound source direction.

10. Troubleshooting

See the “Resources" section of the Programming Guide in the Documentation for more troubleshooting tips.

Audio

·         When running the MicArrayEchoCancellation sample, you should be streaming data before you start the capture loop — for example, play a music file on Windows Media Player.

Depth image artifacts

·         The depth stream images have periodic vertical line artifacts and also radial band artifacts. The magnitude of these artifacts is small enough that they won't interfere with coarse level operations such as skeleton or other object tracking, but they will affect attempts at precise measurement of object dimensions more significantly.  

DirectX Runtime Errors in Samples

• Make sure the driver for your PC graphics card is installed properly.
• Make sure you have Version 9.0c or later of the DirectX runtime.

Speech not working in Shape Game

·         Make sure you have installed the speech prerequisites, including the runtime and the acoustic model.  These downloads are listed in the System Requirements.

 

© 2011 Microsoft Corporation. All rights reserved. Terms of Use | Trademarks | Privacy