|
   
|
(Word version of entire application)
Detailed Discussion of ~70 prior art patents (Word) This document is the first part of the Utility Patent Application. It presents a discussion of the prior art. TITLE:
EASILY DEPLOYABLE INTERACTIVE
DIRECT-POINTING SYSTEM AND PRESENTATION CONTROL SYSTEM AND CALIBRATION METHOD
THEREFOR
APPLICANT:
Erik J. Banning
Cross-reference to related applicationsPriority
is claimed from U.S. Provisional Application filed on May 28,
2004 and from U.S. Provisional Application filed on January 18,
2005. Statement regarding
federally sponsored research or development:
Not applicable. Background of the Invention
Field of the Invention[0001]
The invention relates generally to
devices for making presentations in front of audiences and, more specifically, to
devices and methods for making presentations for which interaction with the
displayed information through direct-pointing is desired or for which verbal
interaction with the audience may be anticipated. Background Art[0002]
Technology
for presenting computer-generated images on large screens has developed to the
point where such presentations are commonplace. For example, the software package POWERPOINT, sold by
Microsoft Corp., Redmond, WA, may be used in combination with a so-called
‘beamer’ to generate interactive presentation material and project for
viewing by an audience. Often, such presentations are held in rooms not
specifically equipped for the purpose of showing presentations, in which case
use is made of a portable beamer in combination with a so-called ‘laptop’
computer. Under these circumstances
the projection surface may be a wall of the room. [0003]
During a
presentation it is desirable for the presenter to be able to move freely in
front of the audience while retaining the capability to interact with the
presentation and point to specific features on the displayed images. It would
also be desirable for the presenter to be able to capture verbal comments made
by members of the audience so as to amplify and/or play them back to the larger
audience. [0004]
In general,
interaction with a computer is often facilitated by pointing devices such as a
‘mouse’ or a ‘trackball’ that enable manipulation of a so-called
‘cursor’. Traditionally, these devices were physically connected to the
computer, thus constraining the freedom-of-movement of the user. More recently,
however, this constraint has been removed by the introduction of wireless
pointing devices such as the GYROMOUSE, as manufactured by Gyration, Inc.. [0005]
Broadly
speaking, pointing devices may be classified in two categories: a) devices for
so-called ‘direct-pointing’ and b) devices for so-called
‘indirect-pointing’. Direct pointing devices are those for which the
physical point-of-aim coincides with the item being pointed at, i.e., it lies on
the line-of-sight. Direct pointing
devices include the so-called ‘laser pointer’ and the human pointing finger.
Indirect pointing devices include systems where the object of pointing
(e.g., a cursor) bears an indirect relationship to the physical point-of-aim of
the pointing device; examples include a mouse and a trackball. It needs no
argument that direct-pointing systems are more natural to humans, allowing
faster and more accurate pointing actions. [0006]
Indirect
pointing devices known in the art include the following. U.S. Patent No.
4,654,648 to Herrington et al. (1987), U.S. Patent No. 5,339,095 to Redford
(1994), U.S. Patent No 5,359,348 to Pilcher et al. (1994), U.S. Patent No.
5,469,193 to Giobbi et al. (1995), U.S. Patent No. 5,506,605 to Paley (1996),
U.S. Patent No. 5,638,092 to Eng et al. (1997), U.S. Patent No. 5,734,371 to
Kaplan (1998), U.S. Patent No.
5,883,616 to Koizumi et al. (1999), U.S. Patent No. 5,898,421 to Quinn (1999),
U.S. Patent No. 5,963,134 to Bowers et al. (1999), U.S. Patent No. 5,999,167 to
Marsh et al. (1999), U.S. Patent No. 6,069,594 to Barnes et al. (2000), U.S.
Patent No. 6,130,664 to Suzuki (2000), U.S. Patent No. 6,271,831 to Escobosa et
al. (2001), U.S. Patent No. 6,342,878 to Chevassus et al. (2002), U.S. Patent
No. 6,388,656 to Chae (2002), U.S. Patent No. 6,411,277 to Shah-Nazaroff (2002),
U.S. Patent No. 6,492,981 Stork et al. (2002), U.S. Patent No. 6,504,526 to
Mauritz (2003), U.S. Patent No. 6,545,664 to Kim (2003), U.S. Patent No.
6,567,071 to Curran et al. (2003) and U.S. Patent Application Publication No.
2002/0085097 to Colmenarez et al. (2002). Each
of the foregoing publications discloses a system for which the 2 dimensional or
3 dimensional position, orientation and/or motion of an object, such as a
handheld pointing device, are measured with respect to some reference coordinate
system using appropriate means. Such
means include acoustic devices, electromagnetic devices, infrared devices,
visible light emitting diode (LED) devices, charge coupled devices (CCD),
accelerometer and gyroscopic motion detectors, etc.. Although for some of the
foregoing devices the reference coordinate system may be positioned close to the
display means, no information on the actual position of the presentation display
with respect to the system is used, causing the resulting pointing action to be
inherently indirect and, hence, less natural to the human operators of such
systems. [0007]
Other
inherently indirect-pointing systems that do not require the position or
orientation of the pointing device to be known include devices such as disclosed
in U.S. Patent No. 5,095,302 to McLean et al. (1992) and US Patent No. 5,668,574
to Jarlance-Huang (1997). The foregoing patents describe indirect-pointing
methods, that do not provide the speed and intuitiveness afforded by
direct-pointing systems. [0008]
Direct
pointing devices are disclosed, for example, in U.S. Patent No. 4,823,170 to
Hansen (1989), which describes a direct-pointing system comprising a light
source, a position-sensing detector located adjacent to the light source and a
focusing reflector that, in one application, is parabolic shaped and is attached
to the pointing device. Additionally, procedures are described to calibrate the
system. In the understanding of current applicant, however, the physical
location of the position-sensing detector needs to be, at least preferably,
adjacent to the display means. The
system disclosed in the Hansel ‘170 patent cannot easily be ported to a room
not specifically equipped for this system. [0009]
U.S. Patent
No. 5,929,444 to Leichner (1999) discloses a system primarily intended for
target shooting practice, but an application as a direct-pointing cursor control
apparatus may arguably be anticipated. The system includes transmitting and
detecting equipment in a fixed reference base and a moveable pointing device.
A calibration routine is described that aligns the detecting and
transmitting means while keeping the pointing device (i.e., a gun) aimed at the
center of the target. The Leichner
‘444 patent does not describe methods or means that allow determination of a
point-of-aim of a pointing device on a target of which the size and/or
orientation have not been predetermined. Consequently,
the system disclosed in the Leichner ‘444 patent is not suitable to be used as
a cursor control means for projection surfaces not specifically adapted to be
used with such system. [0010]
U.S. Patent
No. 5,952,996 to Kim et al. (1999), U.S. Patent No. 6,184,863 to Sibert et al.
(2001) and US Patent Application Publication No. 2002/0084980 to White et al.
(2002) disclose direct-pointing systems where the 3 dimensional position and/or
orientation of the pointing device is measured with respect to sources and/or
detectors, the physical position of which in relation to the display means is
presumed known. Such systems only
work in rooms specifically equipped for their use. [0011]
U.S. Patent
No. 5,484,966 to Segen (1996) , U.S. Patent No. 6,335,723 to Wood et al. (2002)
and U.S. Patent No. 6,507,339 to Tanaka (2003) disclose methods suitable for
direct-pointing that are useful only if the pointing device is physically close
to or touching the display area or volume, for example used with so-called
‘interactive whiteboards’. Some
of the foregoing patents describe appropriate pointer calibration routines.
Such systems are not suitable for presentations where the display surface
is out of the presenter’s physical reach. [0012]
U.S. Patent
No. 6,104,380 to Stork et al. (2000) discloses a direct-pointing system in which
at least the 3 dimensional orientation of a handheld pointing device is measured
by appropriate means. Additionally,
a direct measurement is made of the distance between the pointing device and the
displayed image. However, the system disclosed in the Stork et al. ‘380 patent
does not include methods to ascertain the position and orientation of the
display means relative to the pointing device.
In the foregoing system, these appear to be presumed known. This is also
the case for a system disclosed in U.S. Patent No. 4,768,028 to Blackie (1988),
in which the orientation of a helmet-mounted direct-pointing apparatus is
measured electromagnetically. The
foregoing systems therefore appear to be ill-suited to operate in rooms not
specifically equipped for presentation purposes. [0013]
U.S. Patent
No. 6,373,961 to Richardson et al. (2002) discloses a direct-pointing system
using helmet-mounted eye tracking means. The point-of-gaze relative to the
helmet is measured as well as the position and orientation of the helmet
relative to the display. The latter is accomplished by equipping the helmet
either with means to image sources mounted at fixed positions around the display
or with means to image a displayed calibration pattern. Of course, the foregoing
system relies on sophisticated helmet mounted equipment capable of, among other
things, tracking eye-movement. Moreover, such a system relies on an unobstructed
line-of-sight with respect to the display and a substantially constant distance
from the display to the helmet-mounted equipment. The disclosed invention does
not lend itself to be easily used by a human operator in an arbitrary (not
predetermined) presentation setting. [0014]
U.S. Patent
No. 6,385,331 to Harakawa et al. (2002) discloses a system that uses infrared
technology in combination with image recognition software to distinguish
pointing gestures made by the presenter, without the need for an artificial
pointing device. The disclosed system, however, requires the presentation room
to be set up with highly tuned and sophisticated equipment, and is therefore not
easily ported to a different venue. [0015]
U.S. Patent
No. 6,404,416 to Kahn et al. (2002) discloses a direct-pointing system where a
handheld pointing device is equipped with an optical sensor. In such system
either the display is required to be of a specific type (e.g., a cathode
ray-based display that uses an electron beam) or the displayed image is required
to be enhanced by timed and specific emanations.
When pointed to the display, a handheld pointing device may detect the
electron beam or the timed emanations, and the timing of these detections may
then be used to ascertain the point-of-aim. The disclosed system is somewhat
similar to the technologies used for so-called light guns in video games as
disclosed, for example, in U.S. Patent No. 6,171,190 to Thanasack et al. (2001)
and U.S. Patent No. 6,545,661 to Goschy et al. (2003). Of course, such systems
require either a specific display apparatus or a specialized modification of the
displayed images. Moreover, an uncompromised line-of-sight between the pointer
and the display is a prerequisite for such systems. [0016]
U.S. Patent
Application Publication No. 2002/0089489 to Carpenter (2002) discloses a
direct-pointing system that, in one embodiment, positions a computer cursor at a
light spot projected by a laser-pointer. The system relies on the use of an
image-capturing device to compare a captured image with a projected image. As
such, the system disclosed in the ‘489 application publication makes use of
calibration routines in which the user is required to highlight
computer-generated calibration marks with the laser pointer. The system
disclosed in the ‘489 patent application publication is not unlike a system
disclosed in U.S. Patent No. 5,502,459 to Marshall et al.
(1996). Also, U.S. Patent No. 5,654,741 to Sampsell et al. (1997), U.S.
Patent No. 6,292,171 to Fu et al. (2001), U.S. Patent Application Publication
No. 2002/0042699 to Tanaka et al. (2002) and U.S. Patent Application Publication
No. 2002/0075386 to Tanaka (2002) all disclose systems that can detect a
light-spot using optical means. Such systems specifically generally require the
use of computationally expensive image processing technologies. All of these
inventions require a projection surface with adequate diffusion properties, as
well as some form of optical system with a steady and uncompromised view of the
display area. As such, they limit the freedom-of-movement of the presenter and
place limitations on the position and optical characteristics of the necessary
equipment. Also, in some of these inventions fast and involuntary movement of
the user’s hand may result in a cursor that does not move smoothly or a cursor
that does not perfectly track the light spot, causing possible confusion with
the user. [0017]
Other
pointing systems known in the art may be classified as other than entirely
direct-pointing or indirect-pointing systems. Such systems include one disclosed
in U.S. Patent No. 6,417,840 to Daniels (2002), which is combination of a
cordless mouse with a laser pointer. Although this system incorporates a
direct-pointing device (i.e., the laser pointer), the method used by the system
for interacting with the presentation is indirect (i.e., by means of the mouse)
and therefore does not afford the fast and more accurate interactive pointing
actions provided by some other direct-pointing systems described in some of the
foregoing publications. [0018]
Another
system known in the art that uses both direct and indirect-pointing methods is
described in U.S. Patent No. 6,297,804 to Kashitani (2001).
The disclosed system is a system for pointing to real and virtual objects
using the same pointing device. In the disclosed system, the pointing means
switch between a computer controlled light source (e.g., a laser) and a
conventional computer cursor, depending on whether or not the user’s intended
point-of-aim has crossed a boundary between the real and virtual display (i.e.,
computer-displayed imagery). Various methods are described to establish these
boundaries. Although the computer-controlled light source may be regarded as
pointing in a direct manner, its point-of-aim is essentially governed by the
system operator using an indirect-pointing system such as a mouse. Thus, the
disclosed system does not allow for the desired flexibility afforded by truly
direct-pointing methods. [0019]
Other
systems known in the art include those such as disclosed in U.S. Patent No.
5,796,386 to Lipscomb et al. (1998), which discloses a calibration procedure to
establish the relation between coordinate systems associated with a handheld
device and, in one embodiment, a display unit.
The system disclosed in the Lipscomb et al. ‘386 patent may arguably be
applicable as a direct-pointing cursor control system.
The disclosed calibration routine requires the user to register at least
three 3 dimensional positions of the handheld device with a central processing
unit. The disclosed system does not appear to include calibration methods for
the case where the display unit is out of physical reach of the system operator.
The system is thus not practical for use at an arbitrary venue. [0020]
U.S. Patent
No. 6,084,556 to Zwern (2000) discloses a head-mounted display unit that
displays information governed by the orientation of the apparatus, as measured
by a tracking device. This way, the system creates the illusion of a large
virtual display that is being looked at by the system operator. Of course, the
large display alluded to does not constitute a projected presentation. Also, no
methods are disclosed in the Zwern ‘556 patent to establish the relative
position of the head-mounted apparatus with respect to the display. [0021]
U.S. Patent
Application Publication No. 2002/0079143 to Silverstein et al. (2002) discloses
a system in which the position of a moveable display with respect to a physical
document is established. The ‘143
patent application publication describes calibration routines in which the user
is required to activate a registration button when the moveable display is over
some predetermined position on the physical document. The disclosed system only
relates to 2 dimensional applications and, moreover, cannot be used in
situations where the interaction region is out of the system operator’s
physical reach. [0022]
U.S. Patent
No. 5,339,095 to Redford (1994) discloses an indirect-pointing system where the
pointing device is equipped with non-directional microphone. Also, U.S. Patent
No. 5,631,669 to Stobbs et al. (1997) discloses the inclusion of a
non-directional microphone unit in an indirect-pointing device. |
Contact us at through the Feedback
page with
questions or comments about this web site.
|