Cool Tip: Save time and know when your jobs are done

Part 1 (2017)
1

I’ve been using this for a long time for a lot of things, but it is especially handy with machine learning. I showed this to a friend of mine also taking the course and he seems to like it a lot so I figured I would share it.

I use a service called Pushover, it is a one -time purchase for unlimited push notifications. You pay per platform, $4.99 for all your iOS devices, $4.99 for all your Android devices or $4.99 for all your Desktops, so if you want to use all three it is $15 one-time fee. There are a lot of these services (like Boxcar and so on) but PushOver is by far the easiest, most functional and extremely cheap.

Why is this great?

The API is fantastic and simple. Literally, can be up and running with a new app inside of 5 minutes. Less if you put your coffee down.

Why do I care?

Be notified when a long running machine learning (or any job for that matter) is finished running. It is literally as simple as adding a line like this at the end of your python notebook:

%pushover-ml “Job Finished”

1 second later this arrives

Pushover has support for Python, but I find using curl via bash is much easier as you can create them in one place and call them from anywhere. I have a lot of pushover-xxx scripts that use a custom icon and header depending on the task. You can also use cosmetic formatting with HTML and even create lock screen widgets that update for frequently used updates. The above script is as simple as this:

curl -s \
--form-string "token=APP_TOKEN" \
--form-string "user=USER_KEY" \
--form-string "message=hello world" \
https://api.pushover.net/1/messages.json

Throw that in a sh script changing the app token, user key and setting message to $1 and your set. You can also notify specific devices or just use your default user key that sends to all devices.

Any questions just let me know but it is super easy and extremely helpful. It isn’t limited to machine learning, I use it all over the place for things like notifying me when a backup fails or a mounted cloud drive has to be remounted.

9 Likes
3

Say you are training a model, and it is going to take 4 hours. You can wait and check a few times when it starts getting near 4 hrs time, or just have it send you a push notification when it is finished and not have to worry about it.

2 Likes
4

You should copy this to the beginning of the topic :stuck_out_tongue:

5

I thought the title was pretty descriptive.

1 Like
6

This is a great idea.
Although I should add that its possible to get cheaper(free) solutions for Android and Desktops (and probably even iOS) Maybe something like IFTTT will do it.

7

@harveyslash If you have success with something, could you please write a short instructions. I have heard a lot about IFTTT, but never used it. I’d greatly appreciate it. Maximizing GPU utilization seems to be my new sport :slight_smile:

8

I have a silly question - if you want your Pc to notify your phone, do you have ro pay twice? (Pc & android)
Besides that, great tip! I’ve once tried to make my computer “beep” at the end if trainig, but it didn’t work…

9

You can also easily wire up Amazon SNS with Boto3 (in Python) to text you little notes. It’ll end up costing less than $1/month.

10

I got it to work for free on IFTT.
I am going over the steps briefly here:

IFTT is a service that allows you to do ‘if this then that’ things.

  1. Download IFTT on whatever device(s) you want notifications. Register in IFTTT
  2. go to https://ifttt.com/maker_webhooks and then settings. Enable it.
  3. Head to my applets, and create a new applet.
  4. For service, choose Maker Webhooks
  5. Select Receive a web request
  6. For event name, you can put anything. I put Machine learning Task Complete. Then click create
  7. Then click on ‘that’. It is the BLUE + ICON.
  8. For action service, search and select ‘notification’. There should be just one card, click on it.
  9. Click next, then on review on finish.

How to use?

  1. Go to https://ifttt.com/maker_webhooks and then settings.
  2. You should see a URL that looks something like https://maker.ifttt.com/use/1iY9u-E4QvoVfqmV9ulzq
  3. If you call this url from anywhere, your phone will get a notification
  4. There are several ways to do an http call in python. If you are using jupyter, you can just use curl

@dradientgescent pointed out that IFTTT may be slow an unreliable. Even though I personally haven’t found any issue with it, you have been warned!

Let me know if this was helpful :slight_smile:

7 Likes
11

I think that a tip section should be added to the wiki :slight_smile:

1 Like
12

IFTTT does work but not nearly as clean and easy, and $4.99 one time fee is nothing in the swing of things. It is far more customizable with PushOver than putting something through IFTTT and IFTTT is slow and flakey. Instead of 1 second it can be as long as 15 minutes before you get the push.

13

IFTTT is free, but it is also slow ( as much as 15 minute instead of 1 second), buggy, and lacks a lot of the customization you can do with the pretty html messages. It does work, but to be honest I wouldn’t use it just to save $5. I use IFTTT for other things, but I much prefer PushOver for notifications.

14

It may vary from device to device , but I get notifications instantly(~1 second) on android, and the text is also customisable.

15

My experience with IFTTT is very flakey, and they claim it can take up to 15 minutes for them to check a trigger. I also get triggers that used to work and then all of a sudden stop working for no reason. I have about 30 triggers on IFTTT and every so often I have to go in and delete them and recreate them.

1 Like
16

oh, didnt know of that. Ill add that in my post

17

Yes, it is $4.99 (one-time fee for life) per platform. If you pay for iOS you can have 15 iphones and 10 iPads all for $4.99 one-time. Want Android? Another $4.99 for as many devices as you like. I don’t like the desktop as it requires you to keep a tab loaded in chrome 24/7 to get the desktop notifications. It isn’t a full client, so I just use iOS for $4.99.

1 Like
18

I haven’t used it yet, but PushOver also does widgets that you can have on your lock screen and get updated in real time. For example, you can make a widget that shows the current accuracy and epoch and have it pushing updates regularly to your lock screen. I haven’t messed with it, but pretty cool.

19

Here is an example

20

While we are on the topic of checking time and monitoring, I thought I would share another tip. If you want to know how long each cell took, you can use the magic command

%%time

However, if you install the package ipython-autotime (pip install ipython-autotune), you can print the time of execution for each cell automatically if you include the following in each notebook. I usually put it next to my %matplotlib inline

%load_ext autotime

1 Like
21

The widget functionality I was talking about before you can read about here

you can do some cool stuff like have a dashboard showing (and updating in real time or better yet every epoch/batch) your current epoch, current loss, current accuracy and monitor it from your phone.