Written by Irmak Tevfik on 29 - Sep - 2014

A dive into Microsoft OCR Library: Step by step implementation with Windows Phone 8.1

We recently have Microsoft OCR Library available as a NuGet package to use with our Store Apps. This article will focus on pointing our the cool features available and also 
step by step approach to implement it to a Windows Phone project (including Continuation). So lets begin with the features:

A- On recent chat with development team, they state that the old Bing OCR control has been  deprecated as of March 12, 2014 (msdn Link), and we now have have a client side library (no need for internet access) to use in our development. And I can say that the result is amazing.

B-Currently we have support to 21 different languages listed as: 
Chinese Traditional,Czech,Danish,German,Greek,English,Spanish,Finnish,French,Hungarian,Italian,Japanese,Korean,Dutch,Norwegian,Polish,Russian,Swedish,Turkish,Chinese Simplified,Portuguese.
and it is simplified to have translation in between the recognized texts.

C-Once we start going through the project it will be more clear, but I can say that It is extremely easy to use.

To make this Article easier to read, I have summarized every single Step separately so reader can jump to the section they are interested in:
Step 1: Creating the Windows Phone Project Sample
Step 2: Implement Continuation for FileOpenPicker
Step 3: Implement Microsoft OCR Library

Step 1: Creating the Windows Phone Project Sample

So, I need to point out that, we need to use Windows.Storage.Pickers.FileOpenPicker class to select an image but it is not going to be as easy as Windows apps as we can not use
PickSingleFileAsync on Windows Phone yet. Currently PickSingleFileAndContinue is the method we need to stick on. Underlying problem is that we need to control the state of the application
continuation. 

Lets begin with these simple steps:
1- create a new blank windows phone 8.1 project 
 2- add an extra page named as "TextPage.xaml"
 3- create a folder named Common and add empty classes named SuspensionManager.cs and named ContinuationManager.cs, so at the end our solution will look like


4- in homepage we need to add a <Image/> tag to show the image and a BottomAppBar to have 2 buttons (Open FileOpenPicker and select image and another one to navigate to TextPage to display)
 so our MainPage.xaml will look like as 

<Page
    x:Class="OCR.MainPage"
    xmlns:local="using:OCR"
    mc:Ignorable="d"
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
 
    <Grid>
        <Image Name="image" />
    </Grid>
 
    <Page.BottomAppBar>
        <CommandBar>
            <AppBarButton x:Uid="OpenFileBarButton"
                          x:Name="OpenFileBarButton"
                          Label="Open File"
                          Icon="OpenFile"
                          Click="OpenFileBarButton_Click" />
            <AppBarButton x:Uid="ShowTextButton"
                          Name="ShowTextButton"
                          Label="Show Text"
                          Icon="ShowResults"
                          Visibility="Collapsed"
                          Click="ShowTextButton_Click" />
        </CommandBar>
    </Page.BottomAppBar>
</Page>

5- Once we nagivated to TextPage. we only need a TextBlock to display text results. Go ahead and change TextPage.xaml as

<Page
    x:Class="OCR.TextPage"
    xmlns:local="using:OCR"
    mc:Ignorable="d"
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
 
    <Grid>
        <TextBlock Name="txtBlock" />
    </Grid>
</Page>

So the project outline is ready.

Step 2: Implement Continuation for FileOpenPicker
Lets add the code below to click event of OpenFileBarButton as
private void OpenFileBarButton_Click(object sender, RoutedEventArgs e)
{
    //please note that PickSingleFileAsync not implemented for Windows Phone 8.1 ref http://msdn.microsoft.com/en-us/library/dn614994.aspx
    //need to implement continuation manager http://code.msdn.microsoft.com/windowsapps/File-picker-sample-9f294cba
    FileOpenPicker openPicker = new FileOpenPicker();
    openPicker.ViewMode = PickerViewMode.Thumbnail;
    openPicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
    openPicker.FileTypeFilter.Add(".jpg");
    openPicker.FileTypeFilter.Add(".jpeg");
    openPicker.FileTypeFilter.Add(".png");
    openPicker.PickSingleFileAndContinue();  
}
using PickSingleFileAndContinue will stun the app until the operation is complete. As you can see the method does not return any values.
lets move to our suspensionmanager class and add the following codes to handle application states and resolve any dependencies. 
We will be needing this class for our changes in App.xaml.cs file.

/// <summary>
 /// SuspensionManager captures global session state to simplify process lifetime management
 /// for an application.  Note that session state will be automatically cleared under a variety
 /// of conditions and should only be used to store information that would be convenient to
 /// carry across sessions, but that should be discarded when an application crashes or is
 /// upgraded.
 /// </summary>
 internal sealed class SuspensionManager
 {
     private static Dictionary<string, object> _sessionState = new Dictionary<string, object>();
     private static List<Type> _knownTypes = new List<Type>();
     private const string sessionStateFilename = "_sessionState.xml";
 
     /// <summary>
     /// Provides access to global session state for the current session.  This state is
     /// serialized by <see cref="SaveAsync"/> and restored by
     /// <see cref="RestoreAsync"/>, so values must be serializable by
     /// <see cref="DataContractSerializer"/> and should be as compact as possible.  Strings
     /// and other self-contained data types are strongly recommended.
     /// </summary>
     public static Dictionary<string, object> SessionState
     {
         get { return _sessionState; }
     }
 
     /// <summary>
     /// List of custom types provided to the <see cref="DataContractSerializer"/> when
     /// reading and writing session state.  Initially empty, additional types may be
     /// added to customize the serialization process.
     /// </summary>
     public static List<Type> KnownTypes
     {
         get { return _knownTypes; }
     }
 
     /// <summary>
     /// Save the current <see cref="SessionState"/>.  Any <see cref="Frame"/> instances
     /// registered with <see cref="RegisterFrame"/> will also preserve their current
     /// navigation stack, which in turn gives their active <see cref="Page"/> an opportunity
     /// to save its state.
     /// </summary>
     /// <returns>An asynchronous task that reflects when session state has been saved.</returns>
     public static async Task SaveAsync()
     {
         try
         {
             // Save the navigation state for all registered frames
             foreach (var weakFrameReference in _registeredFrames)
             {
                 Frame frame;
                 if (weakFrameReference.TryGetTarget(out frame))
                 {
                     SaveFrameNavigationState(frame);
                 }
             }
 
             // Serialize the session state synchronously to avoid asynchronous access to shared
             // state
             MemoryStream sessionData = new MemoryStream();
             DataContractSerializer serializer = new DataContractSerializer(typeof(Dictionary<string, object>), _knownTypes);
             serializer.WriteObject(sessionData, _sessionState);
 
             // Get an output stream for the SessionState file and write the state asynchronously
             StorageFile file = await ApplicationData.Current.LocalFolder.CreateFileAsync(sessionStateFilename, CreationCollisionOption.ReplaceExisting);
             using (Stream fileStream = await file.OpenStreamForWriteAsync())
             {
                 sessionData.Seek(0, SeekOrigin.Begin);
                 await sessionData.CopyToAsync(fileStream);
             }
         }
         catch (Exception e)
         {
             throw new SuspensionManagerException(e);
         }
     }
 
     /// <summary>
     /// Restores previously saved <see cref="SessionState"/>.  Any <see cref="Frame"/> instances
     /// registered with <see cref="RegisterFrame"/> will also restore their prior navigation
     /// state, which in turn gives their active <see cref="Page"/> an opportunity restore its
     /// state.
     /// </summary>
     /// <param name="sessionBaseKey">An optional key that identifies the type of session.
     /// This can be used to distinguish between multiple application launch scenarios.</param>
     /// <returns>An asynchronous task that reflects when session state has been read.  The
     /// content of <see cref="SessionState"/> should not be relied upon until this task
     /// completes.</returns>
     public static async Task RestoreAsync(String sessionBaseKey = null)
     {
         _sessionState = new Dictionary<String, Object>();
 
         try
         {
             // Get the input stream for the SessionState file
             StorageFile file = await ApplicationData.Current.LocalFolder.GetFileAsync(sessionStateFilename);
             using (IInputStream inStream = await file.OpenSequentialReadAsync())
             {
                 // Deserialize the Session State
                 DataContractSerializer serializer = new DataContractSerializer(typeof(Dictionary<string, object>), _knownTypes);
                 _sessionState = (Dictionary<string, object>)serializer.ReadObject(inStream.AsStreamForRead());
             }
 
             // Restore any registered frames to their saved state
             foreach (var weakFrameReference in _registeredFrames)
             {
                 Frame frame;
                 if (weakFrameReference.TryGetTarget(out frame) && (string)frame.GetValue(FrameSessionBaseKeyProperty) == sessionBaseKey)
                 {
                     frame.ClearValue(FrameSessionStateProperty);
                     RestoreFrameNavigationState(frame);
                 }
             }
         }
         catch (Exception e)
         {
             throw new SuspensionManagerException(e);
         }
     }
 
     private static DependencyProperty FrameSessionStateKeyProperty =
         DependencyProperty.RegisterAttached("_FrameSessionStateKey", typeof(String), typeof(SuspensionManager), null);
     private static DependencyProperty FrameSessionBaseKeyProperty =
         DependencyProperty.RegisterAttached("_FrameSessionBaseKeyParams", typeof(String), typeof(SuspensionManager), null);
     private static DependencyProperty FrameSessionStateProperty =
         DependencyProperty.RegisterAttached("_FrameSessionState", typeof(Dictionary<String, Object>), typeof(SuspensionManager), null);
     private static List<WeakReference<Frame>> _registeredFrames = new List<WeakReference<Frame>>();
 
     /// <summary>
     /// Registers a <see cref="Frame"/> instance to allow its navigation history to be saved to
     /// and restored from <see cref="SessionState"/>.  Frames should be registered once
     /// immediately after creation if they will participate in session state management.  Upon
     /// registration if state has already been restored for the specified key
     /// the navigation history will immediately be restored.  Subsequent invocations of
     /// <see cref="RestoreAsync"/> will also restore navigation history.
     /// </summary>
     /// <param name="frame">An instance whose navigation history should be managed by
     /// <see cref="SuspensionManager"/></param>
     /// <param name="sessionStateKey">A unique key into <see cref="SessionState"/> used to
     /// store navigation-related information.</param>
     /// <param name="sessionBaseKey">An optional key that identifies the type of session.
     /// This can be used to distinguish between multiple application launch scenarios.</param>
     public static void RegisterFrame(Frame frame, String sessionStateKey, String sessionBaseKey = null)
     {
         if (frame.GetValue(FrameSessionStateKeyProperty) != null)
         {
             throw new InvalidOperationException("Frames can only be registered to one session state key");
         }
 
         if (frame.GetValue(FrameSessionStateProperty) != null)
         {
             throw new InvalidOperationException("Frames must be either be registered before accessing frame session state, or not registered at all");
         }
 
         if (!string.IsNullOrEmpty(sessionBaseKey))
         {
             frame.SetValue(FrameSessionBaseKeyProperty, sessionBaseKey);
             sessionStateKey = sessionBaseKey + "_" + sessionStateKey;
         }
 
         // Use a dependency property to associate the session key with a frame, and keep a list of frames whose
         // navigation state should be managed
         frame.SetValue(FrameSessionStateKeyProperty, sessionStateKey);
         _registeredFrames.Add(new WeakReference<Frame>(frame));
 
         // Check to see if navigation state can be restored
         RestoreFrameNavigationState(frame);
     }
 
     /// <summary>
     /// Disassociates a <see cref="Frame"/> previously registered by <see cref="RegisterFrame"/>
     /// from <see cref="SessionState"/>.  Any navigation state previously captured will be
     /// removed.
     /// </summary>
     /// <param name="frame">An instance whose navigation history should no longer be
     /// managed.</param>
     public static void UnregisterFrame(Frame frame)
     {
         // Remove session state and remove the frame from the list of frames whose navigation
         // state will be saved (along with any weak references that are no longer reachable)
         SessionState.Remove((String)frame.GetValue(FrameSessionStateKeyProperty));
         _registeredFrames.RemoveAll((weakFrameReference) =>
         {
             Frame testFrame;
             return !weakFrameReference.TryGetTarget(out testFrame) || testFrame == frame;
         });
     }
 
     /// <summary>
     /// Provides storage for session state associated with the specified <see cref="Frame"/>.
     /// Frames that have been previously registered with <see cref="RegisterFrame"/> have
     /// their session state saved and restored automatically as a part of the global
     /// <see cref="SessionState"/>.  Frames that are not registered have transient state
     /// that can still be useful when restoring pages that have been discarded from the
     /// navigation cache.
     /// </summary>
     /// <remarks>Apps may choose to rely on <see cref="NavigationHelper"/> to manage
     /// page-specific state instead of working with frame session state directly.</remarks>
     /// <param name="frame">The instance for which session state is desired.</param>
     /// <returns>A collection of state subject to the same serialization mechanism as
     /// <see cref="SessionState"/>.</returns>
     public static Dictionary<String, Object> SessionStateForFrame(Frame frame)
     {
         var frameState = (Dictionary<String, Object>)frame.GetValue(FrameSessionStateProperty);
 
         if (frameState == null)
         {
             var frameSessionKey = (String)frame.GetValue(FrameSessionStateKeyProperty);
             if (frameSessionKey != null)
             {
                 // Registered frames reflect the corresponding session state
                 if (!_sessionState.ContainsKey(frameSessionKey))
                 {
                     _sessionState[frameSessionKey] = new Dictionary<String, Object>();
                 }
                 frameState = (Dictionary<String, Object>)_sessionState[frameSessionKey];
             }
             else
             {
                 // Frames that aren't registered have transient state
                 frameState = new Dictionary<String, Object>();
             }
             frame.SetValue(FrameSessionStateProperty, frameState);
         }
         return frameState;
     }
 
     private static void RestoreFrameNavigationState(Frame frame)
     {
         var frameState = SessionStateForFrame(frame);
         if (frameState.ContainsKey("Navigation"))
         {
             frame.SetNavigationState((String)frameState["Navigation"]);
         }
     }
     private static void SaveFrameNavigationState(Frame frame)
     {
         var frameState = SessionStateForFrame(frame);
         frameState["Navigation"] = frame.GetNavigationState();
     }
 }
 public class SuspensionManagerException : Exception
 {
     public SuspensionManagerException()
     {
     }
 
     public SuspensionManagerException(Exception e)
         : base("SuspensionManager failed", e)
     {
 
     }
 }
Lets move to our ContinuationManager class. We will be having our logic here which will be bind to App.xaml.cs file and will have an interface which we will implement to our page.
Go ahead and replace the content as:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Windows.ApplicationModel.Activation;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
 
#if WINDOWS_PHONE_APP
/// <summary>
/// ContinuationManager is used to detect if the most recent activation was due
/// to a continuation such as the FileOpenPicker or WebAuthenticationBroker
/// </summary>
public class ContinuationManager
{
    IContinuationActivatedEventArgs args = null;
    bool handled = false;
    Guid id = Guid.Empty;
 
    /// <summary>
    /// Sets the ContinuationArgs for this instance. Using default Frame of current Window
    /// Should be called by the main activation handling code in App.xaml.cs
    /// </summary>
    /// <param name="args">The activation args</param>
    internal void Continue(IContinuationActivatedEventArgs args)
    {
        Continue(args, Window.Current.Content as Frame);
    }
 
    /// <summary>
    /// Sets the ContinuationArgs for this instance. Should be called by the main activation
    /// handling code in App.xaml.cs
    /// </summary>
    /// <param name="args">The activation args</param>
    /// <param name="rootFrame">The frame control that contains the current page</param>
    internal void Continue(IContinuationActivatedEventArgs args, Frame rootFrame)
    {
        if (args == null)
            throw new ArgumentNullException("args");
 
        if (this.args != null && !handled)
            throw new InvalidOperationException("Can't set args more than once");
 
        this.args = args;
        this.handled = false;
        this.id = Guid.NewGuid();
 
        if (rootFrame == null)
            return;
 
        var fileOpenPickerPage = rootFrame.Content as IFileOpenPickerContinuable;
        if (fileOpenPickerPage != null)
        {
            fileOpenPickerPage.ContinueFileOpenPicker(args as FileOpenPickerContinuationEventArgs);
        }
 
    }
 
    /// <summary>
    /// Marks the contination data as 'stale', meaning that it is probably no longer of
    /// any use. Called when the app is suspended (to ensure future activations don't appear
    /// to be for the same continuation) and whenever the continuation data is retrieved
    /// (so that it isn't retrieved on subsequent navigations)
    /// </summary>
    internal void MarkAsStale()
    {
        this.handled = true;
    }
 
    /// <summary>
    /// Retrieves the continuation args, if they have not already been retrieved, and
    /// prevents further retrieval via this property (to avoid accidentla double-usage)
    /// </summary>
    public IContinuationActivatedEventArgs ContinuationArgs
    {
        get
        {
            if (handled)
                return null;
            MarkAsStale();
            return args;
        }
    }
 
    /// <summary>
    /// Unique identifier for this particular continuation. Most useful for components that
    /// retrieve the continuation data via <see cref="GetContinuationArgs"/> and need
    /// to perform their own replay check
    /// </summary>
    public Guid Id { get { return id; } }
 
    /// <summary>
    /// Retrieves the continuation args, optionally retrieving them even if they have already
    /// been retrieved
    /// </summary>
    /// <param name="includeStaleArgs">Set to true to return args even if they have previously been returned</param>
    /// <returns>The continuation args, or null if there aren't any</returns>
    public IContinuationActivatedEventArgs GetContinuationArgs(bool includeStaleArgs)
    {
        if (!includeStaleArgs && handled)
            return null;
        MarkAsStale();
        return args;
    }
}
 
/// <summary>
/// Implement this interface if your page invokes the file open picker
/// API.
/// </summary>
interface IFileOpenPickerContinuable
{
    /// <summary>
    /// This method is invoked when the file open picker returns picked
    /// files
    /// </summary>
    /// <param name="args">Activated event args object that contains returned files from file open picker</param>
    void ContinueFileOpenPicker(FileOpenPickerContinuationEventArgs args);
}
#endif
Please note that it is marked with #if WINDOWS_PHONE_APP to run only for windows phone apps. Same logic we will be implementing to our App.xaml.cs file.
Lets move to our HomePage.xaml.cs and implement our Interface IFileOpenPickerContinuable. we will be having method ContinueFileOpenPicker added which will be
triggered as soon as the app is continued.

We need to have the page instance assigned to a static variable, go ahead and add the code below.
public static MainPage Current;
public MainPage()
{
    this.InitializeComponent();
    Current = this;
    this.NavigationCacheMode = NavigationCacheMode.Required;
}

Ok now it is time to use this variable in the most vital place, inside App.xaml.cs where all of the core initiation logic is carried out.
It will be better showing the code needs to be added and explain them:

using OCR.Common;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using System.Threading.Tasks;
using Windows.ApplicationModel;
using Windows.ApplicationModel.Activation;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Media.Animation;
using Windows.UI.Xaml.Navigation;
 
// The Blank Application template is documented at http://go.microsoft.com/fwlink/?LinkId=391641
 
namespace OCR
{
    /// <summary>
    /// Provides application-specific behavior to supplement the default Application class.
    /// </summary>
    public sealed partial class App : Application
    {
        private TransitionCollection transitions;
#if WINDOWS_PHONE_APP
        ContinuationManager continuationManager;
#endif
        /// <summary>
        /// Initializes the singleton application object.  This is the first line of authored code
        /// executed, and as such is the logical equivalent of main() or WinMain().
        /// </summary>
        public App()
        {
            this.InitializeComponent();
            this.Suspending += this.OnSuspending;
        }
 
        private Frame CreateRootFrame()
        {
            if (Window.Current.Content == null)
            {
                Frame rootFrame = Window.Current.Content as Frame;
                // Do not repeat app initialization when the Window already has content,
                // just ensure that the window is active
                if (rootFrame == null)
                {
                    // Create a Frame to act as the navigation context and navigate to the first page
                    rootFrame = new Frame();
 
                    // Set the default language
                    rootFrame.Language = Windows.Globalization.ApplicationLanguages.Languages[0];
                    rootFrame.NavigationFailed += OnNavigationFailed;
 
                    // Place the frame in the current Window
                    Window.Current.Content = rootFrame;
                }
 
                return rootFrame;
            }
            else
            {
                Frame rootFrame = Window.Current.Content as Frame;
                return rootFrame;
            }
        }
 
        private async Task RestoreStatusAsync(ApplicationExecutionState previousExecutionState)
        {
            // Do not repeat app initialization when the Window already has content,
            // just ensure that the window is active
            if (previousExecutionState == ApplicationExecutionState.Terminated)
            {
                // Restore the saved session state only when appropriate
                try
                {
                    await SuspensionManager.RestoreAsync();
                }
                catch (SuspensionManagerException)
                {
                    //Something went wrong restoring state.
                    //Assume there is no state and continue
                }
            }
        }
 
#if WINDOWS_PHONE_APP
        /// <summary>
        /// Handle OnActivated event to deal with File Open/Save continuation activation kinds
        /// </summary>
        /// <param name="e">Application activated event arguments, it can be casted to proper sub-type based on ActivationKind</param>
        protected async override void OnActivated(IActivatedEventArgs e)
        {
            base.OnActivated(e);
 
            continuationManager = new ContinuationManager();
 
            Frame rootFrame = CreateRootFrame();
            await RestoreStatusAsync(e.PreviousExecutionState);
 
            if (rootFrame.Content == null)
            {
                rootFrame.Navigate(typeof(OCR.MainPage));
            }
 
            var continuationEventArgs = e as IContinuationActivatedEventArgs;
            if (continuationEventArgs != null)
            {
 
                // Call ContinuationManager to handle continuation activation
                continuationManager.Continue(continuationEventArgs, rootFrame);
 
            }
 
            Window.Current.Activate();
        }
#endif
 
        /// <summary>
        /// Invoked when the application is launched normally by the end user.  Other entry points
        /// will be used such as when the application is launched to open a specific file.
        /// </summary>
        /// <param name="e">Details about the launch request and process.</param>
        protected async override void OnLaunched(LaunchActivatedEventArgs e)
        {
            Frame rootFrame = CreateRootFrame();
            await RestoreStatusAsync(e.PreviousExecutionState);
            rootFrame.Navigate(typeof(OCR.MainPage), e.Arguments);
            Window.Current.Activate();
        }
 
        /// <summary>
        /// Invoked when Navigation to a certain page fails
        /// </summary>
        /// <param name="sender">The Frame which failed navigation</param>
        /// <param name="e">Details about the navigation failure</param>
        void OnNavigationFailed(object sender, NavigationFailedEventArgs e)
        {
            throw new Exception("Failed to load Page " + e.SourcePageType.FullName);
        }
 
        /// <summary>
        /// Invoked when application execution is being suspended.  Application state is saved
        /// without knowing whether the application will be terminated or resumed with the contents
        /// of memory still intact.
        /// </summary>
        /// <param name="sender">The source of the suspend request.</param>
        /// <param name="e">Details about the suspend request.</param>
        private async void OnSuspending(object sender, SuspendingEventArgs e)
        {
            var deferral = e.SuspendingOperation.GetDeferral();
            await SuspensionManager.SaveAsync();
            deferral.Complete();
        }
    }
}
1- As you noticed we call to the CreateRootFrame method as soon as OnLaunched event triggered. This is the place where we get the current content and call async method RestoreStatusAsync with the state of the execution. 
2- Once we hold the state we will be navigation to the root page where we want to start our app. 
3- Selecting a file will be triggering OnActivated first and will lock the state until we are done.



4- OnActivated is the place where we get the current and previous state and navigate to ContinueFileOpenPicker by using ContinuationManager.
5- OK now lets get our Image data ready. Go ahead and add the following code to the MainPage.xml.cs
‚Äč
private WriteableBitmap bitmapImage;
private void CommandInvokedHandler(IUICommand command)
{
 
}
 
public async void ContinueFileOpenPicker(Windows.ApplicationModel.Activation.FileOpenPickerContinuationEventArgs args)
{
    if (args.Files.Count > 0)
    {
        //get the file
        var file = args.Files[0];
        //get the scaled image
        var props = await file.GetScaledImageAsThumbnailAsync(ThumbnailMode.PicturesView);
 
        //read the image data
        using (var stream = await file.OpenAsync(FileAccessMode.Read))
        {
            //create the bitmap with the image properties
            bitmapImage = new WriteableBitmap((int)props.OriginalWidth, (int)props.OriginalHeight);
            await bitmapImage.SetSourceAsync(stream);
 
            //set the source of the image in Homepage.xaml
            image.Source = bitmapImage;
        }
 
        //set the navigation button visible
        ShowTextButton.Visibility = Windows.UI.Xaml.Visibility.Visible;
    }
    else
    {
        MessageDialog dlg = new MessageDialog("File does not exists!!");
        dlg.Commands.Add(new UICommand("Close", new UICommandInvokedHandler(this.CommandInvokedHandler)));
    }
}
Basically what we did is to get the image file on Continuation we have handled. and read it to a WriteableBitmap.
as we have set it to image.Source, it will be displayed on the screen. Go ahead and give it a try. You will be having it as:


6- As our ShowTextButton is visible now. we can navigate to the page. add Frame.Navigate(typeof(TextPage), bitmapImage) to button click as
private void ShowTextButton_Click(object sender, RoutedEventArgs e)
{
    Frame.Navigate(typeof(TextPage), bitmapImage);
}
This will navigate the Image as parameter to the next page. but we need to capture the Image on TextPage.xaml.cs

7- On TextPage.xaml.cs add the following code to OnNavigatedTo event
if (e.Parameter != null)
{
    WriteableBitmap img = (WriteableBitmap)e.Parameter;              
}
which will be having the image on next page where we are going to use OCR.
Now its time to implement OCR Library

Step 3: Implement Microsoft OCR Library
We have implemented our Continuation, we have passed our image to the next page. Now it is time to use OCR library to look for text in our image.
Lets add our NuGet package to our solution. it should be named as "Microsoft OCR Library for Windows Runtime".



once installed and we try to build it, we should be having an exception as:
WindowsPreview.Media.Ocr does not support the AnyCPU target platform.
as the code developed native we need to be selecting the platform to run on. So I will be changing the debug configuration platform to x86 which will be
building smoothly. 



Now you will be witnessing the magic. Most important class is called OcrEngine in which its constructor takes an enum for language(OcrLanguage). 
so lets declare an instance on the page as English.
OcrEngine oEngine = new OcrEngine(OcrLanguage.English);

This class has only one method for the whole process which is called RecognizeAsync. It only expects parameters Width,Height and the byte array for pixels.
and the return result contains all of the lines, words, line angles,height-width of the words. 

So go ahead and add these methods to the page:

async void LoadData(WriteableBitmap img)
{
    txtBlock.Text = string.Empty;
   OcrResult data = await oEngine.RecognizeAsync((uint)img.PixelHeight, (uint)img.PixelWidth, ConvertBitmapToByteArrayAsync(img));
   foreach (OcrLine item in data.Lines)
   {
       foreach (OcrWord inneritem in item.Words)
       {
           txtBlock.Text += inneritem.Text;
       }
       txtBlock.Text += Environment.NewLine;
   }
}
 
public static byte[] ConvertBitmapToByteArrayAsync(WriteableBitmap bitmap)
{
    using (var stream = bitmap.PixelBuffer.AsStream())
    {
        MemoryStream memoryStream = new MemoryStream();
        stream.CopyTo(memoryStream);
        return memoryStream.ToArray();
    }
}
So simple. it simply extracts every single word from the image and writes it to the TextBlock.
if I try it with image:


and if we try to parse


Source code can be downloaded from :
OCR GitHub
Enjoy Coding:)
comments powered by Disqus