Computer Vision Search: How Can Brands “See” Invisible Fan Photos?

The Power of Skittles: One home-based cookie business shares the fun design idea of another cookie baker who inspires her. How many times will the Skittles logo and packaging be shared without cookie fans feeling like they are advertising?


Check out the Skittles cookies that just popped up in my Facebook News Feed. There are no actual Skittles fruit candies in the cookies — though I personally think that would be an improvement. The Skittles logo and brand packaging are what’s celebrated here.

This is a classic case of a passionate unpaid brand ambassador sharing her love for her favorite candy. But what’s fascinating from a marketing perspective is that nowhere in the original photo from Karen’s Cookies is there any mention of Skittles. We just see them — in super duper enlarged form — and its famous advertising slogan “Taste the Rainbow.”

When the picture is shared by another mom-owned cookie business, Sinful Squares, Skittles are mentioned in the caption. If Skittles were to do a traditional text search for all its brand photos on Facebook, it would easily find the second photo, but never see the first (the more important one since Karen is the “pioneer” advocate).

With computer vision, it is possible for brands to “see” what’s inside of the billions of photos shared on Facebook, which just announced a major redesign of its News Feed to make its picture display larger and more prominent.

Unless there is a special contest or incentive, most consumers don’t bother to tag pictures with the names of brands or products. They genuinely love their favorite brands, for sure, but it seems like “work” or an artificial gesture to type in the names.

A similar case is this viral photo of an amazing wooden sculpture of Pearl Drums. Very few people bother to mention the “Pearl” name when they share this pic with their social networks.

Tree Trunk Music — This rustic tribute to Pearl Drums has made a huge splash on Facebook.

Brands interested in finding their true number of social network ambassadors need to consider these untagged photos that won’t show up in a traditional search.  Pongr’s computer vision technology can be used as a visual search engine for brands to find their logos, packaging and products in photos shared across the Web.

Have you seen any brands make any fun cameos in your friends’ photos lately?

(Pongr’s computer vision technology and mobile Photo Response Marketing platform helps brands turn any of their existing logos, CPGs, visual media and advertisements into an always-on direct response program – and integrates brand photo contests to their CRM. Check out Our Story.)

Breaking News: Pongr Acquisition of Israeli Startup Sightec is a “Computer Vision Technology Game Changer”

Pongr’s acquisition of Sightec’s computer vision technology will soon impact all of our Photo Response Marketing tools. The new super-resolution technology is capable of detecting images at the sub-pixel level, a leap of 10x current image enhancement results. (Pictured above are Pongr Co-Founders Zach Cox and Jamie Thompson)

Consumers snap and send photos of their favorite brands with their mobile phones, and instantly receive a direct response with info about a contest or promotion. No need to look under the hood or think about how everything works — or why our computer vision “knows” the difference between one company logo and another. Whether you are a brand or a consumer using Pongr, we usually don’t want you to think about technology.

But today, for a brief moment, we’re lifting up the curtain.

In a deal announced this morning by Adweek, Pongr has acquired IP from Israeli startup Sightec, a computer vision R&D lab that has perfected the ability to positively identify images at the sub-pixel level even when the camera is shaky or blurry.   Adweek’s Tim Peterson calls the deal an “adrenaline shot” that boosts Pongr’s already formidable image recognition technology.

“Sightec’s technology allows for sub-pixel registration which (Pongr CEO Jamie) Thompson explained would let Pongr detect objects in the foreground and background of an image. “Pongr has been good to date at detecting products when they’re deliberate and promotional, but because of sub-pixel registration, we could pick up products in the background,” he said.


“Sightec also brings image stabilization and enhancement technology that could recognize an object in an blurry or Instagram-filtered image, making the ongoing flood of user-snapped photos less of a headache.”

The Sightec deal adds five more computer scientists to Pongr’s R&D team. Thompson says that Pongr clients will see immediate benefits from the new technology within the next few months.

Here’s a taste of what brands can expect:

“Originally developed for military security camera systems, Sightec’s super-resolution technology is capable of improving image quality 10X over typical image enhancement results… This level of detection requires only 3-5 pixels vs. the 400+ required by competitive systems in use today.


“These significant advantages are achieved through Sightec’s mathematical approach to super-resolution, a contrarian position within the field of computer vision. Pongr will be tuning the technology to make it work for brands and products, and adapting the sub-pixel registration capabilities for wide-scale brand image detection.”

Check out the full text of the Pongr-Sightec announcement below:

The True Universal Language

It wasn’t just Pongr that realized the potential in photo response; Korea found the beauty of it, as well.

Shopping magazines conveniently located inside the trains.

Home shopping has never been easier in Korea. Magazines are delivered to homes carrying hundreds of photos of products that range from clothing to groceries. These magazines are accessible on trains, as well. Internet shopping has spread so rapidly in Korea; photos are the most crucial part of its growth. Kakaotalk, a Korean smartphone chatting application, created ‘Plus Friends’ that users add to their chat list to receive photos or discounts for a ‘Plus Friend’ brand.

Gangnam’s media pole

Korea’s sophistication level of using photos to market products is very high and very convincing. Picture taking has been integrated to almost every electronic product in the country. Gangnam, the city referenced in the song Gangnam Style, has dozens of tall black media poles with large touch screens where people can get transportation information, directions using digital maps, provides free Wi-Fi and even takes pictures of those who are standing in front of the poles. These pictures can be shared online, e-mailed or sent to a personal mobile device.

Photos are efficiently used in Korea for almost every occasion and the integration is recognized as a powerful way to market a product because there is no need for translations or interpreters. It is a universal language that everyone can understand.

At Pongr, we believe that photos can attract more visits to a blog, more buyers to sellers and more fans to a product. We provide the perfect universal playground for brands and fans to have fun while getting to know each other on another level. Pongr will allow you to have a deeper conversation with your fans without any language barriers.Visit to experience the visual playground we share with thousands of fans.

Pongr is Exploding With Passion: Behind the Scenes of the Fast Company Photo Shoot

Fast Company magazine recently explored Pongr's quest to retire QR code technology and asked CEO Jamie Thompson to demonstrate his feelings. Photo Credit: Jordan Hollender. (Click Jamie's mustache to read full story).

Fast Company magazine never settles for boring corporate headshots and their interview with CEO Jamie Thompson (“Forget QR Codes: Pongr Easily Turns Your Photos into Brand Rewards“) was no exception.

Given our focus on tapping into the power of brand logos and iconic advertising imagery, the action scene above may remind you of the Kool-Aid Man mascot, who is constantly bursting through brick walls and shouting “Oh, Yeah!” whenever there is a distress call from thirsty kids.

However, Fast Company editor Jason Feifer had other inspirations in mind.

“Photo recognition isn’t an easy thing to show in a still image. It’s a process, all done digitally, and any attempt to illustrate it would have come out cheesy,” he says. “So I started thinking that, rather than show Pongr’s business, it should show Pongr’s nemesis—the QR code, an instantly recognizable symbol.”

“We thought about various ways we could have Jamie fighting a QR code—he could be stuffing a big one in a garbage can, setting it on fire, kicking it. But in each of those, I just pictured a guy and a square piece of paper drowning in the photo. They all felt empty. Too much dead space. We needed the QR code to be bigger, badder, something requiring a full-on assault. And from there, the answer was obvious: Jamie needed to be busting through one, high-school-football-player style,” adds Jason.

Jamie met Fast Company photo editor Kathy Nguyen at the Manhattan studios of photographer Jordan Hollender, who was charged with the task of bringing out Jamie’s personality — again, no stodgy corporate stuff.

Pongr's Jamie Thompson with his archenemy, the QR Code! (Photos courtesy of Jordan Hollender)

Kathy had a more daunting challenge. Where do you find a printer to churn out billboard-sized QR codes on short notice? And then how do you prop it up?

Large format printers charge about $300 to $400 per sheet for posters that size. And with the plan being to take multiple poses and shots, the budget wasn’t generous enough to go through props like toilet paper. High school cheerleader pep rally banners were also considered, but most of those open and reseal with velcro and that wouldn’t capture the “torn” look the photographer would be striving for.

The giant barcode you see Jamie burst through is actually nine squares of paper glued together. Originally, the plans were to mount the squares onto foam core board. That image would have stood firm, for sure, but it also would have been impenetrable.

Kathy finally settled for firmly stretching the code like a canvas over a wooden frame.

Saving Jamie from countless headaches and a possible broken nose, the prop stylist cut a small hole in the middle of the code for Jamie to stick his head through and then tear a larger opening.

“The thought was that once we ripped it, we couldn’t go back, so we took baby steps before we let Jamie act like the Incredible Hulk,” says Kathy. “I couldn’t believe how animated he was — such a great model. He really brought his A-game!”

To get the “action” look, Jamie tried his best to get a running start behind the QR code, despite the tight quarters in the studio. At one point, he stumbled through the hole and accidentally caused a larger rip than anticipated.

“We were looking for lots of options so we had him running and jumping the whole time. He was absolutely dizzy by the end,” Kathy says.

Pongr CEO Jamie Thompson

Jamie's distaste for QR codes is well known in the image recognition technology universe. (Photos courtesy of Jordan Hollender).

The Fast Company fashion shoot (Did you notice how Jamie is even dressed in the colors of a QR Code) also included poses of the Pongr CEO throwing the shredded code into a trash can and trying to rip apart the remnants like a grizzly bear. Those shots ultimately wound up on the cutting room floor (or whatever magazines call the place they send their outtakes).

“I appreciate his willingness to humor us and keep trying new things. Jamie didn’t need much direction. He kept pushing the boundaries on his own. This was definitely one of my favorite photo shoots,” says Kathy.

Whenever you ask a CEO to act like the Kool-Aid Man, a high school cheerleader and the Incredible Hulk, how could it NOT be?

Fast Company Explores the Power of Pongr

CEO Jamie Thompson hates QR codes and with good reason—they're clunky and limiting compared to the easy and flexibility of image recognition.

Bursting Out: CEO Jamie Thompson recently bantered with Fast Company why he believes Pongr technology tears QR codes to shreds. Photo credit: Jordan Hollender

Q: Who has the World’s Largest Collection of User-Generated Mountain Dew Photos?

A: Is that a rhetorical question?

I like Pongr CEO Jamie Thompson’s humble boast in the February issue of Fast Company magazine about being the curator of more than 16,000 Dew pics (it’s now over 18,000 by the way).  But more importantly, I love how he handled journalist Jason Feiffer’s candid question about what the big deal is — that is, what’s the difference between the way Pongr generates and processes fan photos and a brand just collecting pics on Facebook.

“First of all, with image recognition, you can vary the (direct) response depending on what’s sent in,” responds Jamie. “It gives the brand a level of intelligence that they otherwise wouldn’t get. There’s also a huge data motivation here. We’re entering this wave of so much user-generated content out there, yet so little is actually known about who the customers are.  We do all kinds of computer-related intelligence, both on photos coming into Pongr and across the web.”

“There is a huge data motivation for brands — pockets of data coming in by region. They can ask, ‘How is our product actually doing in the store? How is our product doing in people’s homes? What are people taking photos of, and is it good stuff or bad stuff?  Do we need to adjust our message in real time, our calls to action in real time?  It’s much more than that direct response into a website,” he adds.

Fast Company’s Feiffer has a clever metaphor about the overused catch phrase of “brand engagement.”

“In some ways, I feel like brands today are like 11-year-old boys,” he says, bringing me back to my 6th Grade dance. “A girl will come up and talk to them, but they don’t really know what to say back.”

Find out what Jamie has to say about talking to girls, image recognition, the flaws of QR codes and the future of social and mobile gaming by checking out February’s Fast Company, which is now on the newsstands if you want to get an autographed souvenir copy.

Click here for an exclusive behind-the-scenes look at the Fast Company photoshoot.

Boston Herald: Pongr visual search engine "instantly" tracks fan pics for brands

CEO Jamie Thompson "Pongrs" the iconic Citgo sign in Boston's Kenmore Square. The neon oil company logo, which looms above the Fenway Park outfield, instantly conjures up images of home runs for Red Sox fans. (Source: Boston Herald)

Tool tracks pics of brands
Pongr’s social-network search aids companies

By Donna Goodison
The Boston Herald
June 25, 2011

A new visual search engine lets companies track what consumers think about their brands by analyzing online photos.

Pongr Inc.’s ImagePulse technology scours the billions of Web photos posted from mobile social applications. Its “computer vision” image-recognition technology instantly identifies those that include companies’ brand logos or packaging — whether it’s someone inside an Apple store, showing off a Gucci bag or drinking coffee at Starbucks.

“It’s designed to passively monitor all the brand activists and brand evangelists who are already taking pictures of brands and products when they’re out and about,” said Jamie Thompson, founder and CEO of the Boston mobile marketing company. “As far as we know, it’s the only one of its kind to use computer vision to measure and monitor who’s checking into product as opposed to location.”

Anything openly available on the Web can become part of Pongr’s search index.

“Just like Google indexed text, we’re indexing the world of brand-related photos across any of the mobile social properties that are of interest to us such as Twitpic, yfrog,, Color, Instagram, Hipstamatic and a variety of others,” Thompson said. “We’re adding to that list on a weekly basis.”

Pongr plans to market ImagePulse to large advertising agencies and directly to companies.

“A lot of consumer brands are getting interested in these location-based services and trying to figure out how to fold mobile social engagement into their marketing plans,” Thompson said.

ImagePulse helps brands not only get a sense of whether or not consumers are engaging with their products without being prompted, but what other products they also enjoy. It analyzes the photos, accompanying text and a person’s history to index their buying intentions.

“If your biggest fans are switching between adidas and Nike on a weekly basis, that might tell you there’s not enough loyalty to the brand as you would like,” Thompson said.

ImagePulse discerns a person’s age group and maps their location in real time. It even has a “happiness detector.”

“We have developed emotion-detection algorithms to help us infer what someone’s sentiment might be based on their expression,” Thompson said.

Pongr designs, develops, markets and supports a variety of mobile marketing and image recognition products for brands and advertising agencies. The core Pongr game platform enables brands to leverage image recognition in direct-response marketing programs designed to reward brand advocates for promoting the products they love.

Image recognition technology is useful for scaling and automating the mobile marketing messages that are built into Pongr’s brand picture engagement engine. Some brands build campaigns around consumer engagement and loyalty models, whereas others seek to drive immediate, in-store special offers and redemption, or m-commerce opportunities. If you are a brand or advertising agency looking for ways to leverage your traditional media spend and are considering the use of image recognition, contact us today.

New York Times/VentureBeat: Pongr's ImagePulse "Sees" How You Feel About Brands

ImagePulse is the first visual search engine to measure consumer brand sentiment. Here's a sample ImagePulse snapshot of Starbucks fans who photograph their favorite drinks and share them via social media.

You see it in your everyday social interactions and on the Facebook Walls of your friends. Millions of people are sharing their favorite brands in real time by snapping pics with their mobile phones.

It could be bragging about how they have box seats at Yankee Stadium, or raving about the latest frozen coffee drinks at Starbucks or Dunkin’ Donuts. Or it could be a silly pose at the Apple Store next to the gizmos they covet the most.

As the volume of fan photos exponentially rises, the frequency of tagging (by brand name) dips dramatically. This creates a “search” problem in terms of making it hard for people to find the visual information they may be looking for, or text-based information that would otherwise be associated with visual content. In fact, this visual search problems is why Pongr was originally invented; to design and develop a world-class image recognition system leveraging the founding team’s computer vision and software engineering skills.

Finding and engaging with their most loyal customers reaps huge benefits for brands. But without tagging, these pics are “invisible” if you try to search for them. Enter Pongr’s computer vision technology and ImagePulse product build upon its image recognition system now in use for many mobile marketing and advertising programs.

Here’s what Pongr CEO Jamie Thompson recently shared with The New York Times and VentureBeat regarding the marketing power of tracking down fan photos and determining their Purchase Intent scores:

”If it turns out that large numbers of people are taking pictures of an outdoor advertising campaign, that might tell the brand that it was money well spent.” he says.

“If people in one part of the country (or world) regularly take pictures of a certain product, but they don’t take pictures of the same product in another region, that might tell you that either your product lacks strong advocates in an area or maybe it’s not as readily available. Once you have a sense of where, when and how frequently your brand is being photographed, you can build better brand engagement campaigns based on existing behavior.” Thompson concludes.

To read the rest of The New York Times/VentureBeat coverage by technology reporter Ciara Byrne, click here.

Pongr’s ImagePulse image recognition & brand sentiment analysis is sold as a subscription service to brands and large advertising agencies. The Company also markets mobile and social games the combine brand advocates who take pictures of ads, products and lifestyle moments that enhance a brand’s engagement through the mobile, social, desktop and traditional advertising channels. For more information on how image recognition and Pongr’s mobile marketing platform can amplify your advertising campaigns, Contact Pongr today.

Brand Logo Parody Watch: Froot Loops & Jelly Belly

(When MAD Magazine, Saturday Night Live, Topps Wacky Packages or even the anti-consumerist Adbusters makes fun of your brand, you know you have arrived. “Brand Logo Parody Watch” is an occasional series celebrating some good-natured fun with advertising and brand packaging.)

Wacky Packages pokes gentle fun at Kellogg's Toucan Sam!

As Wacky Packages collector and historian Greg Grant has well documented, the Topps Company has been doing brand satire since 1967. Odds are very high that many of your favorite foods, cleaning and hygiene products have “Wacky” alter-egos.

In Froot Loops lore, “Follow Your Nose – It Always Knows!” was the British-accented Toucan Sam’s catch phrase as he guided jungle creatures and adventurers to “orange, lemon, cherry and other natural flavors.” Toucan Sam has infiltrated American pop culture to the extent that he’s made cameos in two Family Guy episodes.

Here, he’s been reduced to a spokesman for cheap health insurance.  Incidentally, the “OOPs” concept has already been tried in real life by Quaker, which made an OOPs All Berries version of Captain Crunch. The premise is that by “mistake” the manufacturer filled boxes completely with red “Crunchberries” instead of sprinkling them throughout regular batches of Captain Crunch.


The darker side of jelly beans!

WOW. With school bullying taking on epidemic proportions on the national agenda, this is a rather risque theme to have fun with. But Wacky Packs have never shied away from taboo subjects. And Jelly Belly has been known to take some creative risks, too. Sure, getting a product tie-in with the Harry Potter franchise seems like a no-brainer, but does the world really need Booger, Dirt, Vomit and Earwax jelly beans?

And Jelly Belly’s BeanBoozled line of candy includes Skunk Spray, Pencil Shavings and Dog Food.  It’s almost as if Jelly Belly is trying to be more outlandish than the Wacky Package artists!

(Pongr’s mobile picture-sharing game connects brands with their most devoted fans. We also love advertising humor. Do you have a favorite brand spoof to share?  Let us know at

Pongr’s image recognition and mobile marketing platform enables brands to turn their logos and traditional advertising assets into direct-response marketing opportunities. Through the use of picture games, visual search analysis and brand advocates rewards, Pongr makes it easy for any brand to amplify existing advertising spend and connect directly with the most passionate fans for any given brand; those who are taking product and brand related photos across multiple social networking channels. The company uses sophisticated computer vision technology to make logos recognizable across any wireless carrier and 99% of mobile handsets. For more information on our mobile marketing and gaming solutions, contact Pongr today.

Jetray: Concurrent Email Load Generation in Scala

Mail servers. Not very sexy, right? The problem of some IT nerd, or outsourced to GMail. Well at Pongr, mail servers are pretty important to many of our mobile marketing solutions since they currently are the primary way we receive most of these awesome Bruins, Ferrari and Mountain Dew pics (and tons of other brands, of course).

Since they’re so important to us, we need to fully test our mail servers. One type of test we do is load testing, both of our image recognition and email input/response platform. This amounts to generating realistic emails and sending them to our system. For Pongr, “realistic” means an email with a JPEG photo attached, possibly with some text in the subject, and sent to some email address. Just like a user would send us from their mobile phone. We need to generate and send these emails over some period of time, to simulate real users sending photos and then observe & measure how our system performs. It’s important that we understand how picture-texts and emails come in from all kinds of wireless carries.

We searched for tools to help us generate realistic email load, but really didn’t find anything that fit our needs. So we built our own tool, and we’ve now released it as an open source project. Please say hello to Jetray, he’s pretty awesome.

An email load generator needs to be able to send emails at some rate, usually measured in emails/min or emails/sec. We call this email frequency. Another way of looking at this is that the load generator needs to send an email at regular intervals, measured in seconds or milliseconds. This email period is just the reciprocal of email frequency. So if the system sends 6 emails/sec then it sends one email roughly every 166 msec. This frequency (or period) must be variable; that is, the person running the test will want to specify how fast to send email.

A first approach at this problem might be to generate & send emails one right after the other, and if that goes too fast then put the appropriate delay in between them.  But what happens if the time it takes to send one email exceeds the desired email period? Then, my friends, you need concurrency.  You need to be able to start sending an email while another email is already being sent.

There are a lot of ways to run things concurrently. We use Scala a lot at Pongr, and for this task we chose to use the new Typesafe stack, in particular using Akka actors to send mail concurrently. Jetray creates a bunch of actors that can all send email (over an SMTP connection), then each email that needs to be sent is passed to one of those actors.  If one actor is busy sending an email when the next email needs to be sent, it’s just passed to another actor and that actor begins sending it. The result is that Jetray can send email at a much higher frequency (i.e., lower period) than a more naive approach.

We also designed Jetray to be a simple set of components that you can use to build your own email load generator. It’s not a framework. It’s not a stand-alone application with a GUI. Jetray consists of several reusable actors that handle generic aspects of an email load generator, like performing an action at a specified frequency and sending email over an SMTP connection. You then wire these components together, along with your own use case specific actors that actually create the emails, and voila you have your very own custom email load generator.

If you’d like to try out Jetray, the README on github provides a simple example to help you get started. If Jetray is useful for you or you see ways we could improve it, please let us know.