It has been quite some time since AnandTech reviewed cameras or camcorders for their image and video quality. Even amongst professional gadget review sites, there is an absence of a standardized benchmarking methodology for this purpose. Starting with this review, we have decided to use the Handsome Case for evaluating image and video capture gadgets. This kit can be used to determine and analyze many image and video quality characteristics. However, this section will only deal with factors relevant to a security camera in a home setting.
 

Evaluating the Dropcam Echo with the Handsome Case

It is very important for any security camera to have a good performance in low lighting conditions. We performed a quick image quality test of the Dropcam at various light intensity levels. The light temperature was held constant at 5500 K and three light levels were evaluated: 1000 lux, 260 lux, and 15 lux. The evaluation was not extensive and did not included moving objects or resolution measurements. In general, the camera held up well under low light conditions in this static scenario.

One interesting thing is the choice of holding color saturation levels somewhat constant under lower light conditions. Chroma noise becomes prominent at 15 lux, especially noticeable in dark low frequency areas. It is our understanding that most industrial security applications make the design decision to reduce chroma as light intensity falls or simply to move to black and white mode at a given light intensity. The typical security professional would prefer less noise or more color to maintain resolution and detail. Since this is a consumer targeted unit we believe they made the right decision as most consumers likely prefer to see color and will accept the tradeoff of chroma noise. That being said, it might be nice to have a "black and white mode" or some configurability in this area for the discerning low light viewer.

Again, no detailed analysis was performed on the resultant bitstream but the smaller block sizes present in the lower lux scenes seems to indicate that more bits are being used to encode these scenes as the encoder will need to work harder to encode the increased noise patterns.
 

Figure 6A

In Figure 6A, note the low frequency dark area of the color meter. As light intensity is decreased this area contains more noise - especially in the chroma plane. The encoder interprets the noise as detail that needs to be encoded. We don't know the internal workings of the image pipeline, but a 3D noise filter that identifies spatial noise patterns over time would certainly help here. Such filters are present in most of the present day IP camera chipsets. So, future models are likely to have improvements in this area.
 

Figure 6B

In Figure 6B, note the red feather in the scene captured at high lux with a DSLR compared to the screen shots of the unit encoding at 1000 Lux and 15 Lux. The unit loses most of the fine detail at 1000 lux and then works much harder in the lower lux environment likely because a moving noise pattern is present and this noise is attempting to be encoded.
 

Figure 6C

The human eye is trained to look carefully at faces. In Figure 6C, some degradation of the facial resolution is noticeable between 1000 lux and 15 lux, but skin tone is maintained nicely. Note the difference in chroma noise in the low frequency dark area behind each doll.

To summarize the DSP qualities, it can be said that the unit holds up well under differing low light conditions. Chroma levels are maintained with the resultant artifacts noticeable. More configuration options would be nice for low light scenes but the target audience is probably going to be happy with the tradeoffs in the shipped unit.
 

The Cloud Backend Miscellaneous Metrics
Comments Locked

24 Comments

View All Comments

  • MonkeyPaw - Thursday, August 12, 2010 - link

    From my own experience (was burglarized a few months ago in broad daylight), thieves are very bold. They will act like they belong there, and most people won't pay attention, especially if they look official. Unfortunately, all security can be bypassed--even a big dog can be dealt with if someone is determined enough. Your best hope is to make your property less appealing than everyone else's. I don't mean go all ghetto, but smash-and-grab people are pretty lazy, so they will pick the easiest targets first. If only they put their time into productive work, instead of making everyone else's insurance go up every year. >:(
  • jquin6 - Saturday, January 12, 2013 - link

    Criminals taking basic precautions. Did your nanny drop you on your head? Only 40% have some high school or less!
  • mcnabney - Wednesday, August 11, 2010 - link

    The 45GB/mo of upstream should have been mentioned a bit earlier, probably best placed in the cloud section of the article. This product is not really a commercial device and will likely be installed in locations with consumer-grade data connections like cable and DSL (or even wireless like Clear). That type of usage will certainly be noticeable by the ISP. Since this is, as you say, the iPad of IP cameras the likelihood of consumer installs is fairly high and that usage generation will likely come back to bite the customer.

    The cloud usage is really the big drawback, especially when 99.999% of 24/7 data will never ever by viewed. Why pipe it? My first thought was having a two-box solution. One is a camera, the second is small HDD that will store the data. That way the upstream only occurs when requested. It is also easier to haul a little box into court to be used as evidence than to wrangle the cloud. The two box solution could easily be battery backed-up and operate a direct ad-hoc wireless connection.
  • Twoboxer - Wednesday, August 11, 2010 - link

    ^^ Previous comment makes many of the relevant points. Constant streaming is ludicrous.

    The hardware needs to do local motion detection, discriminate between simple lighting changes and real motion, and record only the video from several seconds before the motion detection, through the period of motion, and for several seconds thereafter. The recording should optionally be done to a local device, or to "The Cloud". The choice should be an economic trade-off between owning the local disk hardware, or renting the cloud.

    In either case, real-time remote access needs to be made as easy to implement as their current product apparently is. And a small monthly fee for this service, separate from the above, is reasonable.
  • ganeshts - Wednesday, August 11, 2010 - link

    Twoboxer, The scenario you outline is only one of the possible applications. The solution suggested by you works for that particular scenario.

    When you look at apps like nanny cams (where you keep a watch over the babysitter), constant streaming becomes a necessity, irrespective of motion detection. Agreed, this will not be 24 x 7, as the cam in that particular application can be switched off as soon as the user gets back home.

    Another way to think of this is a webcam without a computer ( though one would definitely hesitate to pay $280 or so for this purpose alone :) )
  • rcc - Thursday, August 12, 2010 - link

    As I mentioned above, the hardware does support local motion detection as you describe. It's a dropcam limitation, most like so they can sell you the various rate plans.
  • andrewbuchanan - Wednesday, August 11, 2010 - link

    I've written several security applications that store video offsite that clients could access anytime through a web browser, one stored mpeg4 clips of motion detection back in 2005, then other one in 2008 stored jpeg images approx every second.

    Neither company really went far with it. The primary technical concern, as somebody else pointed out already, is that most people get 1/2 to 1 Mpbs upload, enough for a couple of cameras max. And that's pinning their upload, in some cases 24x7, something the customer doesn't really want whether they know it or not. And it's bad with dsl/cable providers, worse with cellular or wireless providers. Makes for a nice sales discussion, you either gloss over it but include it in the fine print or scare them with the though of a huge internet bill. Upside is if your house burns down you still have recordings, and in the case of a break in they can't steal the recorder, downside is that your internet connection going down means you lose recordings and that happens all too often.

    My issue personally has always been that the quality of locally stored video can be 30 fps 2MP+ content (these days), or even 30 fps 352x240 (2005), 640x480-2MP by 2008. If it's for security local content just looks way better. I don't think there's enough market for nanny cams to make it worthwhile. Most people don't want cameras in their house, especially ones you can view over the internet.

    And... axis cameras are overpriced, they are nice, but overpriced. But maybe that's why they targeted iphone/apple :-), people who don't mind overpriced.

    Besides the company's I worked with doing this, there were others, so it's not really novel or unique. In most cases I've steered companies to doing something similiar to what I'm doing now, providing local equipment (nvr/dvr) with flash/silverlight/mobile streaming options. Hardware costs are higher, but there's no monthly, it's expandable, flexible, has much better quality, longer recording history, easy on your internet connection, and can still be watched anywhere.

    Downside being it does require someone to actually configure their router which I appreciate is what they are marketing this product as not needing. But is $25/month cheaper than just calling a professional to set up your router for 15-30 minutes?
  • pkoi - Thursday, August 12, 2010 - link

    So in other word that is
    Webcam + h264 compression + nic adapter.
    for ~300$

    In the Cloud ???, Sorry but you inevitably need local storage/server, preferably hidden, with an UPS.
  • ganeshts - Thursday, August 12, 2010 - link

    The review also brings out the USP of the product as the ease of setup and use (even for the non-tech folks).

    Also, the target market is casual monitoring (like nanny cams / looking after pets etc.), not for those really paranoid about security, I guess :)
  • JNo - Monday, August 23, 2010 - link

    It's not the $300 cost of the product that will make them money or not, rather this is like selling printers -the money is in the refills, or in this case the subscriptions.

    "After this, a 7 day recording plan costs $8.95 per month, while the 30 day recording plan costs $24.95 a month."

    At these prices (on par with my mobile phone bill) this is way too expensive for your average jo to be a 'nanny cam'. It is clearly aimed at home security imho. And for most people, a decent holiday is 2 weeks, making the 7 day plan pointless. $25/mth is a lot so the ability to record locally is highly desirable. That's a lot of money for home security based solely on a webcam.

Log in

Don't have an account? Sign up now