Lenovo ThinkSmart Video Conference Camera System User Guide

: June 1, 2024
: Lenovo

Table of Contents

Intro:
Features:
ThinkSmart Cam Specs
AI Function
Whiteboard Awareness
Speaker Tracking
Function:
People Counting
Frequently Asked Questions & Troubleshooting
Optimal Usage Considerations for the Think Smart Camera.
References
Read User Manual Online (PDF format)
Download This Manual (PDF format)

Lenovo ThinkSmart Video Conference Camera System User Guide

Intro:

The Think Smart video conference camera system has a built-in AI NN computing unit, which has advanced smart video conference functions! With the speaker tracking and white board awareness functions that will let the meeting be more focused on the speaker and the content of white board. People counting function can count and display all the number of participants in real time. It will be easier to monitor the dynamics of the entire video conference!

Camera usage / target room size recommendation:

Fully optimised for Microsoft Teams Rooms / Zoom Rooms, as camera is certified for them
Fully optimized for medium – large room usage
Works with Windows 10 PC

Camera Deployment / layout recommendation:

USB cable notes –Only USB2 cable is bundled, bundled cable informally supported
10 mlong cable is also available as option –user can purchase

Where/how to put camera:

on TV by clip, on soundbar by magnet, TV/Wall mount bracket – reference/link to QSG
warning not to put camera on unstable location, as camera may drop and hurt people

Camera angle/direction:

Recommend not to put camera toward glass wall as people behind glass wall is counted by people counting feature.
Recommend not to put camera toward human picture/figure/drawing, as they are counted by people counting feature

SW/FW requirement:

No special device driver required –just works by USB connection to host, as it works with Microsoft Inbox Driver
Firmware inside camera is auto updated by Windows Update

AI function behavior notes:

Auto frame/zoom, whiteboard awareness, people counting – enabled by default
Speakertracking, Picture in Picture disabled by default –need ThinkSmart Manager or Control App to enable it
Whiteboard awareness /speaker tracking does no works when Auto frame/zoom is disabled

Features:

Output format:
AI function support format:
USB2.0
Supports Windows 10
1080P, MJPEG@30fps
720P,MJPEG@30fps
360P, MJPEG, YUY2@30fps

Think Smart conference camera supports below AI functions:

Auto Frame + Auto Zoom
People Counting
White Board Awareness (White Board Awareness + Speaker Tracking +Auto Frame + Auto Zoom)
Speaker Tracking (Speaker Tracking+ Auto Frame + Auto Zoom)

ThinkSmart Cam Specs

Main System Specs

Specification		Smart Conference Camera
Camera Resolution		FHD (1920 x 1080)
Sensor Pixel		2um
Sensor Optical Size		1/1.8”
Lens FOV	Diagonal	125°
Horizontal	100°
Vertical	68°
CUP/SOC		ARM Cortex-A7 processor
GPU/DSP		NPU(MAC)
Output Format		MJPEG, YUY2
Image Format	YUY2	1920×1080/5fps,

Key Parts Specifications

Sensor Specification

Resolution| 8M
Pixel Size| 2um x 2um
Optical Size| 1 / 1.8 “
Active Array Size| 3840 x 2160
CRA| 11°
Maximum Image Transfer Rate| 4K2K : 60fps2250 x 1440 : 60fps1080P : 60fps720p:120fps
HDR| 2-exposure staggered HDR
Lens Specification

Sensor Type| | 1/1.8” 8M Sensor
Focal Length| | 3.7mm
F/NO| | 1.6
Optical FOV| D| 146°
---|---|---
H| 125°
v| 68°
TV Distortion| | <-20%
Relative Illumination| | >50.4%
Structure| | 4 Glass + 3 Plastic

AI Function

Auto-framing + Auto zooming & people counting

System keep detecting human body and count how many people.
Auto framing and zooms the screen for suitable size.
Whiteboard Awareness + Speaker Tracking
Whiteboard is counted as person for auto frame/zoom when people are nearby.
Make sure your space has good lighting. Poor lighting may lead to bad image quality by bad in-camera exposure. Uniform lighting condition will help to provide excellent image quality.

Whiteboard Awareness

The purpose of the whiteboard function is to frame the speaker and the whiteboard and enlarge their images when the lecturer explains the content of the whiteboard in the meeting room, so that the participants can see the content on the whiteboard.

Whiteboard awareness and Auto Frame & Auto Zoom & Speaker Tracking coexist in parallel mode, and people must continue to speak for more than 2 seconds for the function to be running. If a whiteboard is being used in the room, it should be within 5m distance of the camera.

When the whiteboard detection function is activated, the system detects that a person is approaching the whiteboard more than 2 seconds and starts to speak more than 2 seconds. The system will regard the person and the whiteboard as the highest priority “Whiteboard & Speaker ROI” range and enlarge the ROI to match the output image! Make sure your space has good lighting. Poor lighting may lead to bad image quality by bad in-camera exposure. Uniform lighting condition will help to provide excellent image quality.

AI should continuously detect whether the speaker has left the whiteboard or stopped talking for more than 3 seconds, and if so, run the single-person or multi-people tracking auto frame + auto zoom function.

Speaker Tracking

The Speaker Tracking function is a combination of the system’s built-in sound microphone array, DSP sound direction detection, tracking algorithms and uses the picture-in-picture mode under the Auto Framing & Auto Zoom function to place the original picture FOV in the sub image. That will can focus on the speaker and see all the attendees at the same time!

When the auto-framing and auto-zoom functions are activated, the speaker’s speech in the meeting continues 2 seconds or more (the interval does not exceed 0.5sec), will be defined as the “Speaker Tracking ROI”. The “Speaker Tracking ROI” will be appropriate zoom in and show the original FOV image of camera in the sub image of PIP. It will perform image processing for the interest area of the speaker (SPEAKER ROI) and output image resolution as well as crop the image and zoom in and zoom out the multi-people ROI to match the output image resolution.

Note: If there are too few pixels in the FOV, the image will only be enlarged to an appropriate size to maintain the image quality.

If the speaker stops speaking for more than 3 seconds, the video will zoom out to all people in the original image. The sub image is the original FOV image of camera, showing all the participants in the meeting.

Description of special matters:

When the speaker is more than two people and the distance is too close, not more than 50 cm, AI may set the ROI to multi-people ROI, so it will run the multi person auto frame and auto zoom function
Since the sound recognition is the X-axis direction, if the main position of multiple people is the front and back direction and the distance is too close, AI will set the Speaker ROI to the multi-people ROI, and the Auto Frame & Auto Zoom function
The speaker sound volume should be greater than 60dB (normal speaking volume)
The speaker should be less than 5m away from the camera.

Auto Frame + Auto Zoom
Purpose After AI calculation, it can automatically detect and track the participants in the FOV of image. According to the location of person and appropriate ratio to determine the region of interest (ROI) and move and zoom in the ROI to the centre of the output image.

Function:

This function should be able to automatically detect and divide into single person mode and multi-people mode based on the number of people in FOV of image!
The single-person image frame is the smallest ROI, and the single-person image frame is combined into a multi-person ROI according to the number of people. According to the location, the center of the final ROI image is adjusted to the center of the output image and make digital zoom in to the output image!

People Counting

When the people counting function is enabled, the number of people in the meeting room including those entering and leaving the room is counted and the total number of people is instantly updated in the people counting box, which is showed at the top of the output image.

Note: It is recommended to disable AI features like Automated Zoom, Speaker Tracking, and Whiteboard Detection when using Microsoft Teams Room’s Cloud Trammelling.

Configuration
Users can configure AI functions of the Think Smart Cam using either; 1) Think Smart Manager or 2)
Think Smart Cam Control application using Windows 10.

Configure AI Functions using Think Smart Manager
Users can manage the AI Functions of the camera using Think Smart Manager if the device the camera is connected to is added to your organisation in Think Smart Manager. This applies for the Think Smart Hub 500, Think Smart Hub and Think Smart Edition Tiny. To configure the Think Smart Cam:

Find the device within Think Smart Manager, Device Manager area.
Open the device the camera is connected to, by selecting it in the Device List.
- Note : In Open Issues section of the device unresolved issues related to the Think Smart Cam will be shown.
To change the configuration of the Think Smart Cam, find Think Smart Cam in Peripherals list of the specified device and click to manage camera settings:
- AI Functionality
- Auto Zoom/Frame
- Speaker Tracking
- Whiteboard Recognition
- People Count
- Picture in Picture
Users will also find camera details within the Think Smart Manager peripheral list:
- Camera status
- Healthy
- Error
- Firmware Version
- Connection Type
Clicking “Save” will change the configuration of the target Think Smart Cam. A success message banner will be shown.

Configure AI Functions using ThinkSmart Cam Control App
To utilize the ThinkSmart Cam Control App, users can download the application and install it on Windows 10 device. To obtain the application:

Log in to Think Smart Manager web application from your organisation https://console.thinksmart.lenovo.com
Note: For users who don’t have an account, simply create one using the simple self guided wizard. A Lenovo ID is required, this can be obtained at https://passport.lenovo.com .
Once logged into ThinkSmart Manager, Navigate to Applications in the left panel menu.
Download Think Smart Cam Control application
When the application is installed, connect the camera to a Windows 10 PC with the included USB cable.
Once the camera is connected, you can configure the following settings:
- AI Functionality
- Auto Zoom/Frame
  - Speaker Tracking
  - Whiteboard Recognition
- People Count
- Picture in Picture
Clicking “Save” will change the configuration of the target Think Smart Cam. A success message banner will be shown.
Users will also find camera details and open issues within the Think Smart Cam Control App
- Camera status
- Healthy
- Error
- Firmware Version
- Connection Type
- Open Issues

Frequently Asked Questions & Troubleshooting

Question	Answer

Can I change the configuration of a ThinkSmart Cam using the ThinkSmart Cam App or ThinkSmart Manager while the camera is in use?| Yes, configurations can be changed while the camera is in use. This will not stop the video;however it may change the experience for viewers of the video stream.
I see the configuration for people counting as turned on, what does it do?| The ThinkSmart Cam will count the people in the room. In future releases of the ThinkSmart Manager and ThinkSmart Cam app, there will beextended features to leverage the data generated from the people counting functionality.
I don’t have a ThinkSmart Hub, Hub 500 orThinkSmart Edition Tiny, can I use ThinkSmart Manager?| Yes, users create an account and download theThinkSmart Cam App by visiting https://console.thinksmart.lenovo.com
Which Operating Systems are supported by the ThinkSmart Cam Control App?| Windows 10 systems.
The ThinkSmart Cam is not detected by my device, what can I do?| Check the USB connection between the device and ThinkSmart Cam.
Unplug and re-plug all connections.
The Think Smart Cam is not functioning on my device but is detected, what can I do?| Unplug and re-plug all connections.
The Think Smart Cam is displaying an error in either Think Smart Manager or Think Smart Cam Control App Open Issues areas, what can I do.| Check the USB connection between the device and Think Smart Cam.
Unplug and re-plug all connections.
Follow these troubleshooting steps (Windows):

With the Think Smart Cam connected to target device, navigate to the Device Manager (Right click windows start key – > Device Manager)
Navigate to Cameras, find “Think Smart Cam”, as shown below:
Right click on Think Smart Cam and
“Uninstall Device”.
Unplug the Think Smart Cam.
Reconnect the Think Smart Cam.
Check if error is resolved.

Can I use both Think Smart Manager and Think Smart Cam Control App at the same time,on the same device, on a single Think Smart Cam device to configure settings?| No, the Think Smart Manager will be the primary control point for the Think Smart Cam. The Think Smart Cam app will not be permitted to function in this scenario.

Optimal Usage Considerations for the Think Smart Camera.

The Think Smart Camera is a device intended primarily for use in a conference room environment in conjunction with other Think Smart devices and the Think Smart Manager Software. It includes specific Artificial Intelligence (AI) features to support this usage. If used outside of a conference room environment such as in an open plan environment, or in a Work from Home (WfH) environment, the AI features may lead to behaviour by the camera that is not typical of expected function for the device as a webcam, and the function of the device might appear erratic to the user. The device AI is tuned for a conference room and the camera will act based on that configuration if the AI features are enabled. The AI features can be disabled using the Think Smart Camera Control App available at this link https://support.lenovo.com/us/en/downloads/DS542392

Work From Home Usage Example:
Scenario: A WfH environment for a user with other people or pets in the room or moving into/out of the field of view and making noise, or objects in the field of view such as portrait style photographs of people.

In this type of scenario, the camera may try to include other people or objects into its processing for items like Auto Frame / Auto Zoom, Speaker Tracking and People Counting. This may also cause changes in the camera behavior over time, depending on what is happening in the environment. To help mitigate this, use the ThinkSmart Camera Control software to disable features such as Auto Frame / Auto Zoom / People Counting / Speaker Tracking.

Please Note: Changes in camera behavior are not necessarily indicative of a defect. Deactivation of AI features may be necessary to ensure an optimal experience depending on the environment.
Company logo