Automated Tagging of Images on Nextcloud with Tensorflow (for Poets 2)

I take lots of pictures on my mobile phone of nearly everything. Whenever I fuel up my car, I take a picture of my dashboard so that I can remember the date and odometer to be able to calculate my fuel consumption. Another case would be each time I go to supermarket, I take a picture of the receipt provided so that I’ll be able to keep track of my expenses. Going Hiking with friends are also great opportunities to take more pictures. How to  sort out everything? Fortunately we have lots of machine learning tools out there. I chose Tensorflow not because I found it to be the best but simply because it has more buzz around it.

Let’s start by viewing our files in Nextcloud which are synced with my Phone.

In the above example, I manually looked for car dashboard pictures and tagged them  as “Car Dashboard”. Let’s do the same with supermarket receipts, business cars and hiking pictures. I found that we need atleast 30 pictures for TensorFlow to not crash. In case I was feeling lazy, I would just copy/paste the same pictures till I get 30. I am lazy.

Now that it’s all done with like a few, lets download TensorFlow for Poets 2. You may follow instructions from here:

TensorFlow for Poets require you to put your training images into their respective category folder. So we need to download the images we just tagged in the proper way. Nextcloud Offers a WebDAV API as standard. Unfortunately classic python DAV clients would not suffice for our task. We’ll have to create our own methods and overload some.


#git clone
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + '/webdav-client-python')

import webdav.client as wc
from webdav.client import WebDavException

class NCCLient(wc.Client):

We write a function which will download the pictures for each tags specified by their ID and put them inside /tmp/nc/{categoryid}

def download_training_set():
    client = NCCLient(options) 
      for tag in ['2','9','1','3','11','12','14']:
        index = 0
          os.mkdir("%s/%s/" % (options['download_path'],tag))
	for filepath in client.listTag("/files/Nayar/",tag):
	index = index + 1
	filepath = filepath.path().replace('remote.php/dav','')
	local_filepath = '/tmp/nc/%s/%s' % (tag,os.path.basename(filepath))
	if(index > 200):
	except WebDavException as exception:

The method listTag is not present in the default DAV classes so we have to write our own in our extended class

Now that we have all our images in their respective folders, we can run TensorFlow for Poets.


python3 -m scripts.retrain   --bottleneck_dir=tf_files/bottlenecks   --how_many_training_steps=100   --model_dir=tf_files/models/   --summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}"   --output_graph=tf_files/retrained_graph.pb   --output_labels=tf_files/retrained_labels.txt   --architecture="${ARCHITECTURE}"   --image_dir=/tmp/nc/

What’s left now is to download all images from Nextcloud and try to label them with the TensorFlow model. We write the classify_data() function. All images downloaded are renamed as follows /tmp/nc_clasify/{nextcloudfileid}.png.

When the classify function is run on each of the images, they are renamed if the label accuracy is greater then 99.9%. Else they are deleted. The renaming scheme is as follows /tmp/nc_classify/{tagname}-{tagid}-{nextcloudfileid}.jpg

We have to manually check if some images are mislabeled and we simply delete them from the folder. Finally we need to send the labelled images info back to NextCloud. We write the function as follows:

def update_classification():
  client = NCCLient(options) 
  files = os.listdir('/tmp/nc_classify/')
  for f in files:
	match ='(.*)-(.*)-(.*).jpg', f)
	os.remove('/tmp/nc_classify/' + f)

The putTag function is also custom. Please see the gist. And there you go.

Repeat the same process multiple times till your training data get bigger and bigger and it starts to classify more and more images.

Now the next step would be to dig deeper into tensorflow and OCR to automate lists of items I buy from the supermarket and make it generate reports for me.

[Intellectual Property] Part 1: Introduction

Our minds are what drive us everyday to do things. Some tasks we do are  auto-pilot mode and others require us to think. Walking to us seem to be an auto-pilot function but was that always the case? Do you remember the first times you learnt to walk as a baby? It was a very brain intensive process. Our parents taught us part of the solution and our brains adapted the solution to work with our body. Same concept can be said about driving and even talking.

white and green sheetwork on brown surface


Our brains generate ideas to our everyday problems in every aspect of our life. But sometimes we let our auto-pilot take control of our selves and we stop being creative. For example, it’s fine to walk from your chair to your bathroom but to walk from your home to your work 15km apart is still possible but it is not practical. Can you come up with some ideas for it?

  • Put shoes instead of waking naked feet. Much more comfort.
  • Walk early in the morning to avoid high sun temperatures.
  • Find a shorter route on the map. Less tiredness and quicker.
  • Take energy drinks on the way. (I really do not recommend this one)
  • Take a bicycle or a bus instead of using our feet. Less tiredness and quicker.

woman walking on the desert

Now let’s imagine you were the first human to come up with those ideas. Would you prevent others from copying you or would you let them copy you but only if they remunerated you in some ways first?

This is where Intellectual Property (IP) comes in place. You will not be able to protect each and every kind of things which your mind can create. The IP system differs in every country and “by striking the right balance between the interests of innovators and the wider public interest, the IP system aims to foster an environment in which creativity and innovation can flourish” [1].

In the next post, we shall see what the different “Creations of the mind” and what are their legal qualifications. In the meanwhile, can you write down few creations of the mind?



Using old Android Phones as Surveillance Cameras in Home Assistant

Phones get obsoleted very fast nowadays. Meaning you must have phones lying around in your house which are too slow to be used daily. I had an LG E612 which I thought to use as an IP Camera.

I installed “IP Webcam” app from Google Playstore. We need to note down the IP address and Port of the webservice which it starts. Ideally we should fix the IP address on the DHCP of our routers.

Next we configure Home Assistant to fetch data from our phone.

We then restart Home Assistant and voila! We have our livefeed.


Problems encountered:

  • The Android phone seems to be freezing after 5 hours. Maybe it is overheating or something but it doesn’t reboot. It gets stuck on the booting animation. Is it a bug in Cyanogenmod or hardware related?



Playing PUBG Mobile on the move in Mauritius

It takes me 2 hours to get home at Curepipe (Robinson) from Reduit. It’s scandalous. Plus the bus is crammed with people when it reaches Ebene. We need more 173 and 191 Curepipe buses!!! But anyways, to kill some time, I play this game called PUBG Mobile (Player Unknown Battleground).

I know it’s not the most productive thing to do but yeah, it’s relaxing after a day of work.

It’s very much playable on mobile data. The game doesn’t lag at all while the bus is moving. But my friends were curious about how much data does the game consumes. After 2 matches today. I saw that it took about 18MB.

I think we can safely say that a match is about 10MB max. I use the Zeness Pack from Chili in which I get 750MB for Rs 75 + Vat. That means you will be able to have 2.5 winning matches per day for 30 days. I think online playing on mobile is very accessible now.

Do you also play online games in the bus? Please share with us.

Why I switched to a CHILI SIM only #Mauritius

I used to have a Dual-SIM phone (Lenovo S60) with the 1st slot holding a CHILI SIM and the secondary one holding a MyT (ex Orange) SIM. You need to know that the 2 SIM card slots are not created equal on dual-sim phones. The first slot usually has higher bandwidth than the secondary one. Most of the time, the secondary SIM is 2G/2.5G only meaning you can only do phone calls and very slow internet.

For Rs 86.24 (Rs 75 + VAT), you get 750MB of mobile data. This is a lot of data for a mobile phone. The only thing which you would not be able to do is stream videos. But come on. You do have the options of making YouTube videos offline. You can thus watch your videos on the move on the YouTube app.

The best part of it is that the data balance get forwarded to the next month if you buy another package within 30 days.

As if the data package wasn’t good enough, CHILI also give huge amount of free SMS and all calls to other operators occur at a fixed rate.

I was still putting money on my MyT (Orange) SIM just to keep the number. But then I got an iPhone which has only 1 SIM card slot. I had to make a choice. Is mobile data more important or having a phone number which has been yours for almost 5 years?

The Future of Telephony

There are multiple ways to contact a person: WhatsApp, Facebook’s messenger, Twitter App, Gmail, LinkedIn and so on.  When you have mobile data, you have all these methods which are not tied to a single operator.

It’s simply not worth paying MyT loyalty fees. I highly recommend anyone to try the CHILI Zeness Pack.