Samsung Bixby: everything you need to know

Samsung’s Bixby is a software feature introduced alongside the Samsung Galaxy S8 and Galaxy S8 Plus. It’s a digital assistant, like Apple’s Siri and Google's Assistant.

It’s predominantly a voice-controlled AI, aimed at giving you a new way to interact with your phone, bypassing the touchscreen.

You’ll find it in the Galaxy S8 and S8 Plus to begin with, but you can expect it to become a standard feature in many Samsung phones this year and beyond. Unless it falls flat on its face, anyway.

Phone enthusiasts may want to mention S Voice at this point. S Voice was Samsung’s previous attempt at a Siri-a-like service. However, it never fully caught on and its prominence was downplayed after the Samsung Galaxy S5, when Samsung did its best to prune back its phone software in order to make it appear more tasteful.

How does Bixby compare?

One of the problems with S Voice was that it seemed like a “me too” feature. It was like Siri, but not as good.

Learning from its mistakes, Samsung has done its best to make Bixby stand out and seem different. Instead of just asking for directions, cinema times and weather reports, you can ask Bixby to change settings in your Samsung phone.

Google Assistant can change certain settings as well, but as Samsung will know more about the hardware of the devices using Bixby than anyone else, so its assistant is poised to offer deeper control. 

The goal in fact is to let you do anything with Bixby that you can perform using the touchscreen. It’s a HAL 9000.

How do you use Samsung Bixby?

The language for operating one of these digital assistants is already well-established, and Samsung sensibly adheres to these norms. There’s a hardware button you can long-press on the Galaxy S8’s left side to bring up Bixby, or you can say a “wake word” to make it pop-up.

This is just like the “OK, Google” prompt of the Google Assistant, but you can teach the phone a phrase of your choice.

Bixby isn’t just about voice, though. There’s also a Bixby home screen called Bixby Home. This is like Google Now, in that it provides a stream of information cards.

What can Samsung Bixby do?

It’s best to think of Bixby as something with three parts: voice, the home feed and Bixby Vision.

Let’s start with the voice side of Bixby. We’re currently at an early stage of the assistant, and many of the things we take for granted in a digital assistant don’t actually work yet with Bixby.

You can’t ask it how old Madonna is, or find out the largest state in the US — that local information and pub quiz stuff isn’t in place yet. However, you can control a lot of elements within the phone itself.

You can ask Bixby to change to screen brightness to 50%, for example, make it switch on the mobile data hotspot or collate photos taken in a particular location. Most digital assistants primarily look outward, but Bixby looks inwards too.

At present, voice commands work with core phone features, Samsung apps and a selection of Google apps, but support for apps such as Google Play Music will arrive at a later date.

The official line is that “at launch, Bixby’s Voice function will integrate with several Samsung native apps and features including Camera, Contacts, Gallery, Messages and Settings, with the plan to expand its capabilities to include more Samsung and third-party apps in the near future.”

Other commands that already work include rotating photos, using an on-screen image as your wallpaper and beaming a video wirelessly over to your TV (you’ll naturally need a TV that supports this kind of streaming). 

The important element these examples highlight is context. Bixby Voice reacts in relation to what you’re currently doing.

Bixby Home, on the other hand, is where you’ll get the information more commonly associated with this sort of digital assistant software. It’s a feed of (hopefully) prescient info that lives to the left of your Galaxy S8’s home screen.

Until recently this is where you’d normally find Samsung Briefing or Upday, but one with more of an emphasis on news stories. Bixby Home is more diverse, featuring widget-like cards you can toggle on and off, letting you choose what ends up in there.

As well as Samsung Briefing, which continues to exist as a part of Bixby, you can have an S Health step counter, a Facebook module, alarm controls, a calendar widget, one with suggested content from YouTube and another showing off themes for your Galaxy S8.

The more you pack in the less useful Bixby Home may become, but find a loadout that works and you’ll actually end up using it. That’s often the problem with these custom home pages: getting people to use them rather than ignoring them entirely.

Right at the top of Bixby Home is a little ‘eye’ icon, and this launches Bixby Vision. It’s an augmented reality camera feature that scans what the camera sees to find more information about an item or - this is the real aim - get you to buy one.

It does this using optical character recognition, digitizing fonts in the real world, and actually identifying objects just by looking at them. Point the camera at a Nintendo Switch Joy-Con controller, for example, and you can get a price comparison for it.

Alternatively, you can search through Pinterest for similar-looking items/posts. You’ll see buttons for these ‘Shopping’ and ‘Image’ options whenever you tap on an object and Bixby recognizes it as an item.

If this sounds familiar, it's because we've already seen a very similar point, shoot and buy feature on the ill-fated Amazon Fire Phone.

The key to getting this to work is thinking about how object recognition operates. If you’re scanning something with no identifying text and no barcode, the view of the item will have to look similar to the picture used to represent it online.

As a demo of the more specific kinds of recognition possible with Bixby Vision, there’s also a separate wine mode. Yep, it’s used to identify specific bottles of wine. You’ll then see a page detailing suggested food pairings, prices and user (well, drinker) reviews. It should be a neat way to find out if that cut-price Sauvignon Blanc deal at the supermarket really is such a bargain.

What’s the future for Samsung Bixby?

Parts of Bixby are familiar. We’ve seen things like Bixby Home before, and Bixby Vision is a more shopper-driven version of Google Goggles.

The voice side of Bixby is perhaps the most interesting in terms of how it could influence the way we use our phones. However, like most custom Samsung Android additions, it’s limited by support.

It currently works with a fairly small selection of third-party apps, although Samsung hopes to change this. “Our plan is to eventually release a tool (in SDK) to enable third-party developers to make their applications and services Bixby-enabled easily,” Samsung says.

Of course, this is not going to make Bixby ubiquitous because it will always be a feature for Samsung devices alone. This will make it less appealing for app developers already struggling to juggle their priorities.

However, if nothing else these elements of Bixby will provide inspiration for future versions of Android, as a fistful of Samsung software additions have in the past.

This is not Samsung’s aim as such, though. It wants Bixby to become an alternative to Amazon’s Alexa, and as such you can expect Bixby’s reach to extend to future Samsung smart home devices, Samsung TVs and other connected gadgets later this year.