Snacking Robots: Applying Artificial Intelligence to Oreos

Artificial Intelligence, Computer Vision | by

    Trial and Error: HERB the Robotic Butler demonstrates how NOT to separate Oreo cookies from the Oreo creme at Carnegie Mellon University in Pittsburgh.

    When I was a kid, my sister would bite off all of the Oreo creme filling and put the two chocolate sandwich cookies back in the package for the next unsuspecting family member.

    Toying around with the Oreo filling is an American family tradition, as evidenced by the ongoing creme etchings on the Oreo Facebook page, including an Oreo Daylight Savings Clock, an Oreo NASCAR Race Flag, and an Oreo Pirate Eyepatch.

    As part of their new “Cookie or Cream?” campaign, Oreo has contracted some top scientists to solve a daunting century-old problem:

    “Over the last hundred years, almost literally billions of productive man-hours have been lost to people splitting apart OREO cookies by hand and eating the part they prefer. In an attempt to make the world a better and more efficient place, OREO has contracted the world’s best roboticists, artists and tinkerers to create machines that will do the work for us. Take a look at the future of eating OREO cookies.”

    Robotics experts at Carnegie Mellon University were up to the challenge. Through trial and error, their HERB (Home Exploring Robotic Butler) robot learned to twist and separate the cookie without breaking it, scrape off the excess filling and polish off any residue so the cookie is completely clean and ready for consumption by those who prefer their Oreos naked.

    The engineering challenge was immense. Think about it: Even just a teeny bit more pressure in those robotic hands and the cookie gets pulverized. Watch for yourself how HERB handles the challenge:

    The first part of the robot’s challenge involves computer vision and artificial intelligence. The robot must “see” the object and positively identify it as a cookie. It must coordinate its computer vision with the movement of its robotic arm and robotic hand.

    Pongr’s ongoing computer vision research, which just was dramatically enhanced with the acquisition of Israeli startup Sightec (see AdWeek), focuses on identifying brand logos, products and CPG packaging — but does not involve any cookie separation algorithms.

    We do however, have a voracious appetite for Oreo filling — with or without the help of human or robot butlers!

    (Check out the streaming Oreo Fan Photo Gallery on Pongr.)

    Computer Vision Search: How Can Brands “See” Invisible Fan Photos?

    Computer Vision, Image Search | by

      The Power of Skittles: One home-based cookie business shares the fun design idea of another cookie baker who inspires her. How many times will the Skittles logo and packaging be shared without cookie fans feeling like they are advertising?

      Yummy.

      Check out the Skittles cookies that just popped up in my Facebook News Feed. There are no actual Skittles fruit candies in the cookies — though I personally think that would be an improvement. The Skittles logo and brand packaging are what’s celebrated here.

      This is a classic case of a passionate unpaid brand ambassador sharing her love for her favorite candy. But what’s fascinating from a marketing perspective is that nowhere in the original photo from Karen’s Cookies is there any mention of Skittles. We just see them — in super duper enlarged form — and its famous advertising slogan “Taste the Rainbow.”

      When the picture is shared by another mom-owned cookie business, Sinful Squares, Skittles are mentioned in the caption. If Skittles were to do a traditional text search for all its brand photos on Facebook, it would easily find the second photo, but never see the first (the more important one since Karen is the “pioneer” advocate).

      With computer vision, it is possible for brands to “see” what’s inside of the billions of photos shared on Facebook, which just announced a major redesign of its News Feed to make its picture display larger and more prominent.

      Unless there is a special contest or incentive, most consumers don’t bother to tag pictures with the names of brands or products. They genuinely love their favorite brands, for sure, but it seems like “work” or an artificial gesture to type in the names.

      A similar case is this viral photo of an amazing wooden sculpture of Pearl Drums. Very few people bother to mention the “Pearl” name when they share this pic with their social networks.

      Tree Trunk Music — This rustic tribute to Pearl Drums has made a huge splash on Facebook.

      Brands interested in finding their true number of social network ambassadors need to consider these untagged photos that won’t show up in a traditional search.  Pongr’s computer vision technology can be used as a visual search engine for brands to find their logos, packaging and products in photos shared across the Web.

      Have you seen any brands make any fun cameos in your friends’ photos lately?

      **
      (Pongr’s computer vision technology and mobile Photo Response Marketing platform helps brands turn any of their existing logos, CPGs, visual media and advertisements into an always-on direct response program – and integrates brand photo contests to their CRM. Check out Our Story.)

      Taking User-Generated Content By The Horns

      Advertising, Computer Vision, Direct Response Campaigns, Image Recognition, Mobile Marketing, Photo Sharing | by

        The Pongr site features user-generated images on a day to day basis.

        You walk into a store, see a pair of sneakers you love, try them on, buy them, snap a picture, and share it across many of your favorite social networks. Sound familiar? I’m sure we’ve all done something along the lines of this, whether it is clothing, food, or literally anything else. We are living in a time when pictures speak louder than words, and are pleasing on the eyes too.

        This user-generated content is priceless to us here at Pongr, just as it should be to every brand out there. Brands are finally taking advantage of the authenticity that is coming through social networking channels, and using it as a way to connect with new and existing consumers. When a consumer has the opportunity to participate in a brand’s community and create their own content, it opens up a level of trust and communication that cannot be reached through one way advertising.

        Crowdsourcing campaigns are being launched left and right by big brands, calling on the consumers to get involved, whether it be by sending in pictures, videos, stories, or simply voting on something.

        This is a big trend currently in the food industry, seen in brands like Pepsi, Doritos, and most recently, Taco Bell. “Feed the Beat: SXSW 2013” is Taco Bell’s new campaign, working to leverage user-generated content with Twitter and Vine. Taco Bell is calling upon its fans to share photos and videos of themselves enjoying live music in order to create a “Rockumentary.”

        The ideas for these crowdsourcing campaigns are endless, and are becoming an increasingly normal habit in today’s media driven world.

        Here at Pongr, we work on many user-generated photo campaigns, such as the recent UNREAL Candy “Unjunk Your 2013 Sweepstakes.” All that participants had to do was email or picture text a photo of UNREAL Candy for a chance to win a $10,000 shopping spree at Target. (Check out the UNREAL photo gallery here.)

        Pongr user Ariel M. shows off her favorite UNREAL Candy!

        With Pongr’s image recognition technology, we are able to take this user-generated content to the next level by offering direct response photo marketing built especially for brands. It’s as easy as snapping a pic of your favorite brand, sending it in, and immediately getting a response from that brand with information about a contest or a promotion. This direct response technology allows users to truly feel the love from brands they are using on a daily basis, while allowing brands to receive vital information about their consumers, all at the same time.

        Wouldn’t you like to get a response from one of your favorite brands about a picture you posted on the web? Well, at Pongr, we make that possible.

        **
        (Pongr’s mobile Photo Response Marketing platform and image recognition technology helps brands turn any of their existing logos, CPGs, visual media and advertisements into an always-on direct response program – and integrates brand photo contests to their CRM. Check out Our Story.)

        Breaking News: Pongr Acquisition of Israeli Startup Sightec is a “Computer Vision Technology Game Changer”

        Computer Vision, Engineering, Featured Posts, Image Recognition, Image Search | by

          Pongr’s acquisition of Sightec’s computer vision technology will soon impact all of our Photo Response Marketing tools. The new super-resolution technology is capable of detecting images at the sub-pixel level, a leap of 10x current image enhancement results. (Pictured above are Pongr Co-Founders Zach Cox and Jamie Thompson)

          Consumers snap and send photos of their favorite brands with their mobile phones, and instantly receive a direct response with info about a contest or promotion. No need to look under the hood or think about how everything works — or why our computer vision “knows” the difference between one company logo and another. Whether you are a brand or a consumer using Pongr, we usually don’t want you to think about technology.

          But today, for a brief moment, we’re lifting up the curtain.

          In a deal announced this morning by Adweek, Pongr has acquired IP from Israeli startup Sightec, a computer vision R&D lab that has perfected the ability to positively identify images at the sub-pixel level even when the camera is shaky or blurry.   Adweek’s Tim Peterson calls the deal an “adrenaline shot” that boosts Pongr’s already formidable image recognition technology.

          “Sightec’s technology allows for sub-pixel registration which (Pongr CEO Jamie) Thompson explained would let Pongr detect objects in the foreground and background of an image. “Pongr has been good to date at detecting products when they’re deliberate and promotional, but because of sub-pixel registration, we could pick up products in the background,” he said.

           

          “Sightec also brings image stabilization and enhancement technology that could recognize an object in an blurry or Instagram-filtered image, making the ongoing flood of user-snapped photos less of a headache.”

          The Sightec deal adds five more computer scientists to Pongr’s R&D team. Thompson says that Pongr clients will see immediate benefits from the new technology within the next few months.

          Here’s a taste of what brands can expect:

          “Originally developed for military security camera systems, Sightec’s super-resolution technology is capable of improving image quality 10X over typical image enhancement results… This level of detection requires only 3-5 pixels vs. the 400+ required by competitive systems in use today.

           

          “These significant advantages are achieved through Sightec’s mathematical approach to super-resolution, a contrarian position within the field of computer vision. Pongr will be tuning the technology to make it work for brands and products, and adapting the sub-pixel registration capabilities for wide-scale brand image detection.”

          Check out the full text of the Pongr-Sightec announcement below:

          Image Recognition: Dream It and Do It

          Artificial Intelligence, Augmented Reality, Computer Vision, Mobile Marketing | by

            We use intelligent technology every day to make our lives simpler and more manageable. From the software that predicts what we really meant when we type an incomprehensible text message on our smartphones to the algorithms that allow us to search the web for exactly what we need, these intelligent tools are everywhere.

            Brands that successfully harness intelligent technology produce fun, engaging marketing that is guaranteed to surprise and delight. One of the most powerful aspects of Pongr’s technology is our robust image recognition and computer vision capabilities, unmatched in the field. Our software gives brands the freedom to implement almost any concept they can think of in order to supercharge their direct response marketing.

            ImagePulse Mobile Photo Uploads (Infographic)

            Computer Vision, Sentiment Analysis, Shopper Marketing | by

              This May, Pongr unveiled its new visual sentiment image recognition tool, ImagePulse. ImagePulse uses computer vision technology to track the endless stream of brand photos spread across cyberspace. ImagePulse is able to recognize logos and products whether or not the brand is tagged in a photo, which helps brands take advantage of volumes of fan-generated content that previously didn’t show up on their radar.

              This summer, we’ve been tracking mobile photo uploads and gathering data about popularly photographed brands. Here are some of our more interesting findings!

              Boston Herald: Pongr visual search engine "instantly" tracks fan pics for brands

              Computer Vision, Image Search | by

                CEO Jamie Thompson "Pongrs" the iconic Citgo sign in Boston's Kenmore Square. The neon oil company logo, which looms above the Fenway Park outfield, instantly conjures up images of home runs for Red Sox fans. (Source: Boston Herald)

                Tool tracks pics of brands
                Pongr’s social-network search aids companies

                By Donna Goodison
                The Boston Herald
                June 25, 2011

                A new visual search engine lets companies track what consumers think about their brands by analyzing online photos.

                Pongr Inc.’s ImagePulse technology scours the billions of Web photos posted from mobile social applications. Its “computer vision” image-recognition technology instantly identifies those that include companies’ brand logos or packaging — whether it’s someone inside an Apple store, showing off a Gucci bag or drinking coffee at Starbucks.

                “It’s designed to passively monitor all the brand activists and brand evangelists who are already taking pictures of brands and products when they’re out and about,” said Jamie Thompson, founder and CEO of the Boston mobile marketing company. “As far as we know, it’s the only one of its kind to use computer vision to measure and monitor who’s checking into product as opposed to location.”

                Anything openly available on the Web can become part of Pongr’s search index.

                “Just like Google indexed text, we’re indexing the world of brand-related photos across any of the mobile social properties that are of interest to us such as Twitpic, yfrog, img.ly, Color, Instagram, Hipstamatic and a variety of others,” Thompson said. “We’re adding to that list on a weekly basis.”

                Pongr plans to market ImagePulse to large advertising agencies and directly to companies.

                “A lot of consumer brands are getting interested in these location-based services and trying to figure out how to fold mobile social engagement into their marketing plans,” Thompson said.

                ImagePulse helps brands not only get a sense of whether or not consumers are engaging with their products without being prompted, but what other products they also enjoy. It analyzes the photos, accompanying text and a person’s history to index their buying intentions.

                “If your biggest fans are switching between adidas and Nike on a weekly basis, that might tell you there’s not enough loyalty to the brand as you would like,” Thompson said.

                ImagePulse discerns a person’s age group and maps their location in real time. It even has a “happiness detector.”

                “We have developed emotion-detection algorithms to help us infer what someone’s sentiment might be based on their expression,” Thompson said.

                Pongr designs, develops, markets and supports a variety of mobile marketing and image recognition products for brands and advertising agencies. The core Pongr game platform enables brands to leverage image recognition in direct-response marketing programs designed to reward brand advocates for promoting the products they love.

                Image recognition technology is useful for scaling and automating the mobile marketing messages that are built into Pongr’s brand picture engagement engine. Some brands build campaigns around consumer engagement and loyalty models, whereas others seek to drive immediate, in-store special offers and redemption, or m-commerce opportunities. If you are a brand or advertising agency looking for ways to leverage your traditional media spend and are considering the use of image recognition, contact us today.

                Brand Logo Parody Watch: Froot Loops & Jelly Belly

                Computer Vision, Image Search | by

                  (When MAD Magazine, Saturday Night Live, Topps Wacky Packages or even the anti-consumerist Adbusters makes fun of your brand, you know you have arrived. “Brand Logo Parody Watch” is an occasional series celebrating some good-natured fun with advertising and brand packaging.)

                  Wacky Packages pokes gentle fun at Kellogg's Toucan Sam!

                  As Wacky Packages collector and historian Greg Grant has well documented, the Topps Company has been doing brand satire since 1967. Odds are very high that many of your favorite foods, cleaning and hygiene products have “Wacky” alter-egos.

                  In Froot Loops lore, “Follow Your Nose – It Always Knows!” was the British-accented Toucan Sam’s catch phrase as he guided jungle creatures and adventurers to “orange, lemon, cherry and other natural flavors.” Toucan Sam has infiltrated American pop culture to the extent that he’s made cameos in two Family Guy episodes.

                  Here, he’s been reduced to a spokesman for cheap health insurance.  Incidentally, the “OOPs” concept has already been tried in real life by Quaker, which made an OOPs All Berries version of Captain Crunch. The premise is that by “mistake” the manufacturer filled boxes completely with red “Crunchberries” instead of sprinkling them throughout regular batches of Captain Crunch.

                   

                  The darker side of jelly beans!

                  WOW. With school bullying taking on epidemic proportions on the national agenda, this is a rather risque theme to have fun with. But Wacky Packs have never shied away from taboo subjects. And Jelly Belly has been known to take some creative risks, too. Sure, getting a product tie-in with the Harry Potter franchise seems like a no-brainer, but does the world really need Booger, Dirt, Vomit and Earwax jelly beans?

                  And Jelly Belly’s BeanBoozled line of candy includes Skunk Spray, Pencil Shavings and Dog Food.  It’s almost as if Jelly Belly is trying to be more outlandish than the Wacky Package artists!

                  (Pongr’s mobile picture-sharing game connects brands with their most devoted fans. We also love advertising humor. Do you have a favorite brand spoof to share?  Let us know at tips@pongr.com)

                  Pongr’s image recognition and mobile marketing platform enables brands to turn their logos and traditional advertising assets into direct-response marketing opportunities. Through the use of picture games, visual search analysis and brand advocates rewards, Pongr makes it easy for any brand to amplify existing advertising spend and connect directly with the most passionate fans for any given brand; those who are taking product and brand related photos across multiple social networking channels. The company uses sophisticated computer vision technology to make logos recognizable across any wireless carrier and 99% of mobile handsets. For more information on our mobile marketing and gaming solutions, contact Pongr today.

                  Mobile Marketing: QR Codes vs. Image Recognition

                  Computer Vision, Image Search, Mobile Marketing | by

                    The notion of using 2D barcodes, or QR codes, in mobile marketing campaigns seems to be generating a lot of chatter in the advertising industry, but what’s the real story behind these blotchy little black and white critters of product packaging? And, how will QR codes really work in an economy where many people are still using feature phones? From a 25,000 ft. view, 2D barcodes sound pretty bad-ass… put a medium sized, funky looking barcode on a magazine ad or billboard and customers will instantly be able to connect to your brand’s offering.  Sounds great, right? Not so fast.

                    In almost every meeting, clients and partners ask us what we think about these nifty little codes and we always respond with the following: “2D barcodes have done well in geographically small, highly saturated, and highly controlled or monopolized advanced markets, such as South Korea and Japan, but the U.S. wireless market is a lot different and requires an understanding of what actually makes the codes works. More importantly, you need to understand the nuances of how the barcodes are interpreted by mobile devices to calculate how successful (or not) the technology will be for your specific needs. There are a couple of important reasons why 2D barcode campaigns do well in South Korea and Japan, but will cause significant issues among consumers in the U.S. and most of Europe.  It has to do with standardization–or, in the case of the U.S. and Europe, the lack thereof.  Standardization of a minimum set of hardware requirements on camera phones, standardization of the barcode reader software on the phones, etc. etc.” While the technology behind 2D barcodes is old and quite simple, the barriers to entry for the mobile consumer are high.  2D barcodes were invented in the 1990′s and have since been applied to the mobile market to varying degrees of success and failure.  Basically, they are a souped-up version of the traditional barcode you see on just about everything.  As one can trap more data in 2D barcodes, this makes a great argument for marketers looking to apply things such as: hyperlinks to mobile sites, scan-to-call, scan-to-coupon, or other product/service related content.

                    Bridging The Gap Between TV, Print, In-Store, Mobile & Social!

                    All Categories, Artificial Intelligence, Computer Vision, Image Search, Mobile Marketing, Our Favorites | by
                      Pongr TV Image Recognition

                      As some of you know, we have been re-focusing our efforts on building the most accurate and robust brand image recognition system to allow mobile users to submit pictures and get brand promotional campaign messaging in response.

                      Most people generally assume that this is limited to static images, however we’ve been conducting a number of internal experiments and customer proofs-of-concept around the notion of linking video into the system.  This morning, while waiting to join the daily Pongr development scrum call, a TV ad on CNBC caught my attention and I decided to give it a try.

                      Pongr matched the shot originating from my BlackBerry and returned the search results from Pongr’s automated mobile delivery service.  The entire process happened in the span of about 15 seconds on a normal mobile carrier connection–not WiFi, and with a bad signal at that.

                      While our image recognition system still requires more data to make it as useful as possible for people, we are pleased that the application to TV is working as well as the application to print, billboard, in-store, etc.  We have an even more robust system still under development, which will provide additional forgiveness to less than optimal user generated images, and can’t wait to give that a shot.

                      To learn more about Pongr’s image recognition technology and interactive advertising software for brand and agency partners, please visit our B2B solutions website.