By Adnan Khan

Power Platform:
Mobile App To Count Objects In An Image

This article I can relate with my previous article in which I used one of the AI builder component – >Form Processing. In this article I will use another very interesting Power Platform AI Builder component and develop a Canvas app to use it on any mobile device.

This kind of app can be used for Retail Merchandising Audits where field service agents make sure that the product is positioned in the way that makes it most appealing to the consumer also how many and what items are on the shelf. Automatic input will reduce human errors and will save the agents time.

Another real time scenario could be where a company which provides items on rental and upon return staff count all the items to make sure that customer has completely returned all the items back to them or their warehouse. Suppose these items numbers are usually bigger and its time consuming for staff to count and input data in the system every time customer returns it .

I will show in this article that how easy it is to develop such application with power platform with few easy steps.

Lets start the implementation!

1. Train AI Builder Component

I will first train AI builder’s Object Identification component by providing the images of my objects. It’s important to upload enough images to train AI model. At least 15 images should be uploaded for each object. With fewer images, there’s a strong risk that your model will learn concepts that are just noise, or not relevant. Training your model with more images should increase the accuracy. provide images with different angles, backgrounds, object size, groups, and other variants.

You need to have Power Platform license. If you don’t have you can start by creating office 365 trial on this URL. Https://Portal.office.com. You can follow instructions in below URL to setup power apps environment. https://docs.microsoft.com/en-us/power-platform/admin/create-environment

  • Once you have all setup you can click on AI Builder and then Build. Select Object Identification.
  • Provide any name to your Builder
  • There are options available on first screen. We will start with Common Objects
  • If you already have entities and objects in CDS database then you can select “Select from database” option. Otherwise just select + Add new object and create new objects. In this example to keep it simple I will create new.
  • Here you can add objects you want to identify by AI builder. I have given few names below.
  • When you click next it asks you to upload images.
  • Now upload images for each objects as mentioned above at least 15 for each object.
  • Interesting fact is that if you will upload same image with a different name. AI builder will detect it and will not upload.
  • After upload you will see a screen where you have to tag object in each image.
  • AI builder will also show you the number of images tagged for each object.
  • Click next and click on Train. It will take a few minutes.
  • After training you will see AI builder will show you its performance in percentage. This performance score indicates how well the model did on the images you uploaded.
  • You can click on Quick Test button to quickly test your configured AI builder.

I have provided one image and you can see it has detected all three objects and also showing the accuracy percentage.

2. Use in a Canvas app for any mobile device

  • To develop a canvas app using this builder click on use model button or create an app from scratch by clicking Apps on main left menu of power platform.
  • Select Insert and then select Object Detector from AI Builder menu
  • Select the newly created AI Builder component from AI models list
  • After selection. You will see a detect button. Click on detect button and then select any image to test your builder.
  • Now to see the count select Insert from the menu again and then from Gallery menu select Vertical or Horizontal gallery. It will drop a gallery on the canvas app form. Click on one of the image thumbnail in the gallery and then delete it.
  • One Single line of Code : Click on the gallery on canvas app form. After selecting the gallery type the circled text as mentioned in image below. The Object detector is the object name of your AI builder which can be seen on left hand side under screen one. You will see counts appearing on the gallery.
  • Click on File and Save this app. You can test this app by clicking play button.
  • This app can be used on mobile devices by downloading Power Apps Mobile app on the mobile device IOS or Android https://powerapps.microsoft.com/en-us/downloads/
  • Follow the instructions from here to configure this app on mobile.

https://docs.microsoft.com/en-us/powerapps/user/run-canvas-and-model-apps-on-mobile.

  • Try it yourself.

Featured Insights

Stay Informed with ITKnocks'
Latest Technical Articles

More Articles

Let's Shape the Future Together!

Ready to shape the future of your business? Connect with ITKnocks, your catalyst for innovation. Let’s collaborate and transform possibilities into reality. Contact us, and let the possibilities unfold! 

Get In Touch