The Pros and Cons of Mobile Testing

There are two distinct environments where mobile testing gets consistently brought up by organizational leadership: academic and non-profit/business.  In this article I’ll explore the pros and cons of both.  But first, what is mobile testing?

Mobile testing can broadly be defined as any electronic assessment performed on a non-traditional, mobile computing device.  At this point in the mobile lifecycle, I might also add that mobile testing is new and not particularly vetted.  Yet due to “mobile” being the L&D modalité du jour it’s not uncommon to for one manager or another to pop into the training office and ask about iPads.  It’s even more apparent in academia, where getting an institution’s name in the paper often has images of kids tapping and swiping away on a tablet.

Assessment Types

Assessment doesn’t have to mean test in the traditional sense.  Of course an assessment can be multiple choice, fill-in-the-blank, etc. where the test-taker sits and answers questions with or without resource materials.  This type of information memorization has its place in certain disciplines.  But what about active assessment?  Consider assessing groups of participants on a large project or having a participant demonstrate knowledge by performing an action.  This is where mobile fits in.

Where Mobile Fails

As of this writing, the two types of devices usually defined as mobile are the tablet and smartphone.  Now, not all tablets are created equally, and many perform identical functions as a desktop computer (like the Windows Surface).  But for standard tests on standard mobile devices, the following are concerns:

• Many testing platforms use Flash, which most mobile devices cannot access.
• Smartphones, and to a lesser extent, tablets, are typically used in public areas.  That means wireless access can be unreliable and spotty.  Or, using a 4G cellular connection can pull huge amounts of data.
• The screens are much smaller and difficult to read.  Analyzing an image, typing a response or tapping a hotspot could be problematic.
• Being in a public area may mean breaks in concentration.  (The sound of steaming lattes breaks mine, at least.)

Mobile Opportunities

Think about the second assessment type I outlined above.  Mobile is perfect for demonstration and active assessment.

• Employees can take a live tour of a facility and demonstrate understanding of processes by taking pictures of correct objects.
• Groups could produce an easy Prezi with sound and video and let users tap and swipe to explore.
• In a machine shop with stations, students watch a demonstration video of a procedure.  Then, students record themselves on the device performing the same procedure for review.

A mobile device is not just a mobile computer.  As with any modality, tools should make sense for the learning activity.

There’s Trouble With Online College, But the NYT Misses the Mark

Thanks to my former colleague Ben Miller for tweeting this article.  You can look at the original article from The New York Times here.

There is trouble with online college, certainly, and as a relatively new method of education (including fully-online degrees), there are significant obstacles through which the education community needs to break.   It’s not had the pedagogical study that decades of face-to-face, traditional university settings have had.  And pedagogical methods are changing as rapidly as the latest iPad lands in a freshman dorm.  So let’s take a look at where the NYT oversimplifies and misses the mark on its criticism of online college.

By opening with Stanford’s much-lauded Massive Open Online Course (MOOC), it fails to distinguish the significant differences between those low-risk courses and enrolling in an online course as a fee-paying student.  90% attrition isn’t surprising, considering:

• Students enrolled in a MOOC have very little incentive to succeed; there is no “punishment” like a poor grade, being forced to repeat a course, or pay back financial aid.
• Rarely do students enroll in a MOOC for anything other than professional development or personal interest.
• Many hiring, employment, and business experts discourage including non-credit courses on a resume.

Don’t get me wrong – I love TED as much as the next gal.  But I wouldn’t consider myself a “student” when I’m watching an instructional video or free lecture.

Let’s turn to actual, high-stakes online courses.  The article goes on to rightly point out the serious barriers that factor into being a true online student.  “The research has shown over and over again that community college students who enroll in online courses are significantly more likely to fail or withdraw than those in traditional classes, which means that they spend hard-earned tuition dollars and get nothing in return. Worse still, low-performing students who may be just barely hanging on in traditional classes tend to fall even further behind in online courses.”

These retention figures are a norm across the country.  But universities and small community colleges have tools and research to institute policies and help students achieve.  Students who can simply sign up for an online class, get an instructor who is not prepared for the online environment, and fail because they lack technological capabilities is a tragedy.  At a minimum, all online programs should offer the following basics:

• Rigorous frontloading and required training for faculty who want to teach online.  An instructor should go through a comprehensive training program that includes practice, mentoring, and, if unsuccessful, should not be allowed to teach online.
• Restriction on enrollment for students who lack online skills.  These skills could be tested in an assessment center prior to enrollment.  It should test advanced technical skills; time management; and understanding of differences between online and face-to-face environments as starting points.  Further, students should be required to have a modern, well-functioning computer with several “back-ups” in case of hardware failure (such as the library).  Equipment failure is not an option for the online student.
• Continuing education and development for both faculty and students as technology changes.

Instructors should be trained on the concept of online presence.  Managing students through weekly e-mails isn’t enough.  Video, VoIP, imagery, discussion boards, and journals are all simple and easy-to-use tools that move the instructor away from an anonymous facilitator and back into the role of professor.  (And this just scratches the surface in terms of what instructors should and should not be doing to instruct a successful online course).

Certainly, online courses are a cash cow for many institutions (especially the for-profits who pre-package dry course material that can be cycled through dozens of instructors without changing).  The initial expense of training instructors and preparing students for the online environment would be easily overcome with a highly successful online program that attracts students globally.  What it takes is significant planning, analysis, and administrative support – which not all colleges are, unfortunately, willing to do.

10 Colleges Most Creatively Using Mobile Technology

I stumbled across this blog from a retweet.

I’ll admit, I thought the whole mobile thing was going to be a passing fad. I didn’t even have a cell phone in high school, nor did I start texting until after college. (My 17-year-old niece probably thinks I’m a dinosaur.) It wasn’t too long ago that I picked up an iPad and an iPhone, and the designer in me became enamored with the possibilities.

The universities in this blog piece were focusing more on small steps to move into the mobile sphere. Accessing courses and grades on your phone and mobile newspapers are definitely innovative. But what Bangladesh Open University is doing is phenomenal: “One method utilized in its classrooms blends SMS with TV and/or radio for a multimedia experience encouraging digital discussion while soaking up recorded lessons.”

More and more universities are realizing the benefits of moving classes online. These 10 universities have demonstrated the value of adding mobile into the mix.

Get Yammer working for you

News: Yammer has just been acquired by Microsoft.  Interesting development!

It hasn’t just been one client who’s mentioned that their internal social media site is a dying wasteland.  After the initial excitement of creating a profile, following others, and making a first post, sites like Yammer peter out.  Why?

• You already communicate with coworkers daily via e-mail.
• There’s rarely enough time during the workweek to research and post interesting new insight about your industry.
• You can’t chat about dramatic topics like you do on Facebook.
• If nobody else uses it, why should I?

These issues are similar to any workplace adoption difficulty.  If you want your internal social media site to flourish, it’s going to take the same steps used to manage a procedural or cultural change in the office.  Try these tips on for size:

• Post company announcements on the site.  Send a company e-mail with a link to the internal site.
• Ask for weekly volunteers to post industry news (and give them a headstart by providing some topics).
• Post monthly pictures from around the office, such as company parties, conference presentations, and everyday interaction.
• Encourage employees to post project success stories as well as lessons learned with difficult clients.
• Ensure that department managers are commenting on posts at least weekly.  Buy-in from leadership is critical.
• For minor company decisions, consider using the polling functionality.  (Where should we move the vending machine?)
• Let employees use the site as a professional sounding board. Don’t discourage participation by heavy moderation, within reason.

If you try any of these tips, I’m eager to hear the results.

A long trip

I’m trying to re-caffeinate after a long journey with my client.  You know those times when your flight comes in late at night but you still gotta go to work early the next day?  Well, that’s me right now.  Honestly, they need to add more hours to the day.

Not that the journey wasn’t worth it.  I’ve learned a lot in the past few days about this client, but most importantly, I’m reflecting on the criticality of developing solid objectives for -any- initiative, whether it be training or a large PR campaign. 

Many projects start with a business goal (whether these are well-written or appropriate is for another blog).  We then target project objectives to match that business goal.  But many times, objectives are written without real analysis.  Consider:

This PR campaign will inform the public about our new widget.

OK, that’s a start.  But there’s nothing measurable at all about this objective.  Who is the public?  Why do they need to know about the widget?  What does ‘inform’ mean?  Let’s try this:

Employees will understand our new non-smoking policy.

This suffers from the same problems at the previous objective.  What do the learners need to do in order to ‘understand’ the policy?  Being able to re-write a policy verbatim certainly isn’t the objective.  Let’s try this:

After training, employees will no longer smoke within 20 ft. of the entrance of the building.

See the difference?  I added ‘after training’ for clarity, but we’re focusing on actual observable and measurable results.  I can measure knowledge with a test, but I can do it better by measuring how many employees are still smoking in an inappropriate area. 

See if you can think of ways to improve the PR campaign objective from before.  I’ll bet you learn something about writing good objectives.

 

Share this:

As the Christmas season winds down, Best Buy’s abject failure to fill thousands of orders for its most important two months of the year caught the attention of Forbes contributor Larry Downes.

He writes about his abysmal experiences dealing with the big-box retailer.  As he shops, an employee approaches him in an attempt to upsell an Internet service, but does so in an irritating, obstructive tone. “As a sometime business school professor, I could just imagine the conversation with the TV department manager the day before.  “Corporate says we have to work on what’s called up-selling and cross-selling,” the clerk was informed in lieu of actual training on either the products or effective sales.  “Whenever you aren’t with a customer, you need to be roaming the floor pushing our deal with CinemaNow. At the end of the day, I want to know how many people you’ve approached.”

But this is hardly customer service.  It’s actually getting in the way of a customer who’s trying to self-service because there’s no one around who can answer a basic question about the store’s confusing layout.  It’s anti-service.”

From my perspective, and the perspective of those who commented on this article, the training practices at Best Buy, well, stink.  I remember the years leading up to Circuit City’s ultimate downfall: friends of mine employed by the electronics giant would tell us about the slashing of hours, emphasis on upselling and not on keeping the stores clean, the removal of good and knowledgeable employees to be replaced with high school kids.

Training and retaining talent obviously isn’t Best Buy’s #1 issue.  But how many times have you gone into a Best Buy only to be given lackluster customer service because of uninformed employees?  I can tell you my own story.

I purchased a TV for my husband’s birthday during Black Friday weekend.  Unfortunately, the day after we set it up, the thing died (blown capacitor, if I recall).  I’d had to borrow a truck to pick up the TV, so I called the store’s Geek Squad line to get information about how I should return or at least bring the TV in to be repaired.  “No problem,” the guy on the phone helpfully told me.  “Just bring in your receipt to the store and we’ll arrange pickup.” Great!  I wouldn’t have to bother my aunt to use her truck again — they’ll pick it up!

Receipt in hand, I go into the Best Buy and wait in the customer service line.  A woman at the counter listens to me explain what I’d heard on the phone, then turns to another associate to figure out how to enter this into the computer.  The associate turns to me and tells me I have to call their 1-800 number to arrange pickup.  I explained that I’d called Geek Squad before, and that I wouldn’t have driven to the store on my lunch break if I’d known I could just call.  He shrugs and tells me the number again.

Lo and behold, after 20 minutes on hold in the Best Buy parking lot, the phone representative tells me what I’d first heard: bring the receipt to the store and have them arrange it; they don’t set up pickups over the phone.  I tell her that they don’t believe me, so she insists I ask a manager.  I hang up with her and end up getting in touch with a manager.

This starts an entirely new debacle because I picked up my TV and they didn’t deliver it, he couldn’t arrange for it to be picked up.  If I’d have known there was a delivery option to start with, I’d have done that! (I asked the girl who sold me the TV initially if Best Buy does delivery; she conferred with another associate who told me they don’t).

In this case, a total of four frontline customer service representatives gave me conflicting and false information.  Perhaps these employees were purely seasonal for the holidays, but shouldn’t they still be able to execute basic policies and procedures?

This ended up longer than I anticipated.  But it just shows that effective, meaningful training is essential for any company, even one as large as Best Buy.

Interface Inspirations From Games

When designing interfaces for WBT, practical controls exist in almost every module: Play, Stop, Pause, Next, etc. Although quite functional, these controls are passive options that help us drudge through the material. What many designers are missing is the opportunity to use an accepted part of WBT to add interactivity.

As we look for inspiration in interface design (from vehicle gauges to smart phones), an excellent source that creates interactive interfacing as its lifeblood is the video game industry. Video games live and die on interfaces – they’re one of the most essential functions because it’s how the player interacts with the environment.

Consider FarmVille, a wildly successful Facebook game that probably pops up far too many updates from your friends than you care to see. It has a simple interface that informs you of your farm’s status. Click the button with an image of a hoe, and you start to plant. Easy, huh?

One of the simplest interfaces in the world is one that doesn’t appear at all! The ever-popular Angry Birds displays a slingshot with a bird waiting patiently in the band – and an environment of breakable-looking objects. It doesn’t take more than a moment to start slinging birds across the screen with the touch of your finger, watching your score get ever-higher.

In more advanced games, facial expression technology is a relatively new feature. With Elder Scrolls IV: Oblivion, you can interact with your world by using ‘Persuasion.’ When speaking with characters, they will react to what you say by appearing happy, upset, neutral, or possibly confused. You can use those cues to make the right choices (to get items for cheap, or to avoid a fight, for example).*

How can we draw inspiration from these games? Let’s think about sales training.

• One goal is to say the right thing to make a customer satisfied. What happens when the wrong thing is said? You might get an unhappy customer. Consider branching scenarios where, depending on the learner’s input, a customer becomes happy or upset in real time.
• Or, create icons of products to show to a customer, rather than having the learner choose a line of text out of a multiple-choice option. Perhaps each part of the product is clickable, allowing the learner to describe features to the customer (some relevant, some not).
• What about the interface that doesn’t appear? A video or image of a customer browsing products appears with several clickable objects – the products, a disorganized shelf, a mess on the floor, and the customer. The learner must do something, and likely clicking the customer first to interact with her is a good option.

Next time you’re sitting down to a game of Call of Duty, or watching your aunt play Bejeweled, watch how interactions unfold. You might be surprised at how games mirror instructional design principles.

*N.B. Skyrim’s rocking the socks off Oblivion.

Disengaged in the workplace

Gallup reported today that 71% of Americans are disengaged at work.

As an HPT practitioner, it’s difficult to pinpoint a distinct cause for a national issue. Despite the overall trend, it comes down to individual issues at each organization. Disengagement can be caused by a business’s culture, how it handles employee motivation, career advancement, etc. But, as Gallup reports, disengagement creates problems for organizations that gravely affect the bottom line. In sum? If your employees aren’t engaged, you need to bring in an expert. Slapping a one-size-fits-all solution certainly won’t work.

A little local news

We usually envision training as the teaching of ideal performance in an organization. Apparently, ideal performance at Salt Lake County is dishonesty:

“At a training session soon afterward, the bailiff recorded Warnick telling those at a training session, “We are not going to show them our errors” and also instructed those at the session to leave some information out of incident reports, according to the charges.”

How ‘Occupy’ affects training

What does the socioeconomic ‘Occupy Wall Street’ movement have to do with employee development? Everything.

T&D pointed me to this Business Week article about employee engagement. Why aren’t employees motivated? Writes Dov Seidman, “We cannot “motivate” engagement (or innovation, growth, or succession for that matter); instead, we must inspire the kind of outcomes we want by rooting ourselves in a set of values, being in the grip of an idea worthy of dedication and commitment, connecting around a meaningful and shared purpose, and aligning around a common, deep, and sustainable set of human, societal, and environmental values.”

Although ‘Occupy’ is a conglomeration of many ideas about economic injustice, the overarching theme I’ve interpreted is that of fairness. And it’s not just the people who are willing to camp out in a park to get the discussion going – it’s any employee you talk to in a depressed economy. In a survey, 83% of employees reported that they either do not discuss career goals at work, or only have the discussion once per year. Many have become disillusioned as they watch benefits dwindle while news report indicate millions in bonuses for some CEOs.

I don’t want to start a discussion on the so-called 99% vs. 1%. That argument is for someone else to make. What’s important here is an employee’s perception – whether he or she is valued, connected, and has a meaningful purpose, as Seidman wrote. Are businesses using those key attributes when developing performance goals? Should they? Why or why not?