Tips and Tricks to improve workflow

Part 2 (2017)
1

Here are couple tips from my experience:

#1:
To use Sublime or other GUI editors to edit code on remote/headless Linux from your local laptop - I had a good luck with “Mountain Duck” software: https://mountainduck.io/
It installs agent on Windows or Mac and then you can use your favorite local editor to work on source code and switch to terminal to run it. It also allows to use Finder (Mac) or Explorer (Windows) to navigate over your remote Linux space.
Big time and nerves saver.

#2.
Install pudb - excellent debugger to debug and inspect Python code with pseudo-graphical window on terminal.
http://heather.cs.ucdavis.edu/~matloff/pudb.html
Helps to go step-by-step through the code, inspect variables and understand structures.
Great learning tool.

Gleb

3 Likes
2

I’d also suggest checking out VS Code, it is quickly becoming the Sublime Text replacement since Sublime Text is nearly abandon-ware. There are some plugins for remote sync vi a ftp & sftp. You can also use X11 forwarding and forward VS Code to your desktop from the headless server. If you are on Windows, Xming works great.

VS Code also works helps solve #2 as well since it has the ability to debug python code.

You also get linting, code completion, and snippets (Sublime can do these as well, but I’ve given up waiting for Jon to release 3.x much less support it).

VS Code Python Support

1 Like
3

I wonder about performance implication of remote debugging, especially GPU-powered Python code.
I’ve experimented with Komodo remote debugging and abandoned it due to performance hits.
pudb has no performance hits.

4

Interesting stuff. I’ve been actually doing all the development local on a GitHub repo (that I forked from the class one) using small datasets. Only when I feel like I want to run larger experiments do I push my code to master and then pull down to the p2 instance. Most of the rest of the coding is just on the iPython notebook after that. If I do need to edit the script files, I’ll just edit them using the iPython web interface (seems you can open up .py files and just edit them). If there are significant changes I’ll git to commit back to repo.

What you guys are doing seems nice though… I may try it.

1 Like
5

Thank you for the tips Gleb!

Is there any chance that you know of a tool like mountain duck for linux systems?

6

Mountain Duck allows sane way to navigate and access Linux file systems from Windows/Mac.
Do you mean you want to do it in reverse?

Gleb

7

No, my desktop is Linux as well :slight_smile:
but it still sound like a good tool for editing text on server, however mountain duck doesn’t seem available for linux.

8

I really like using vim in a terminal (plus tmux) to edit text on a server - well worth learning IMHO.

If you want to use a graphical editor on Linux, you could just use https://github.com/libfuse/sshfs . Or run your editor over x-windows.

1 Like
9

I also have a Linux desktop. I’ve been using rmate to remotely edit files. It is designed for the TextMate editor, but Sublime Text and VS Code have extensions that can work. I use it with VS Code. Here’s roughly how to set it up:

First, when SSH-ed into your server, get rmate installed. I prefer to use the bash version so there’s no ruby dependency. You can install it by copying these two lines to the terminal for VS code:

sudo wget -O /usr/local/bin/rmate https://raw.github.com/aurora/rmate/master/rmate
sudo chmod a+x /usr/local/bin/rmate

(by the way Ctrl-Shift-V pastes into the terminal, something I only discovered recently and that saved me countless painful right clicks)

Now from your local computer, you’ll need to create a reverse SSH tunnel so that the server rmate code can communicate with you. There are plenty of ways of doing this, but if you’re connecting using a private key to an ubuntu server, it will be something like this:

ssh -T -R 52698:localhost:52698 -i your-pvt-key.pem ubuntu@your-instance.com

where your-pvt-key.pem is your private key file for connecting to the server, and your-instance.com is the address of your server. This creates a reverse tunnel (-R) from port 52698 on your local machine to port 52698 on your remote server. rmate defaults to port 52698, although this can be changed by modifying configuration files. You will need to open port 52698 in your security group on AWS if you’re using AWS.

If you are using VS Code as your local editor, you must install the “Remote VSCode” extension, and run the server locally in VS Code (type F1 and then enter “Remote: start server”) so that everything is set up. Sublime and TextMate have similar settings.

Now you’re all set up! Just type rmate yourfile.py on the server and it will open the file on your local computer in VS Code to be edited. You can open as many files as you like in this fashion, and you get a full blown text editor with no lag when editing these files :slight_smile:

EDIT: I don’t see why this shouldn’t work from Windows too, provided you can set up an SSH tunnel using PuTTY - I’d be curious to see if anyone gets it right!

4 Likes
10

Thanks for the tip @Dario. I set up forwarding this evening, it is so great to have a good text editor remotely!

11

Thanks! I’m using sublime, followed the instruction here

12

I really like features like autocompletion, quick documentation lookup etc when editing files. Have been using pycharm IDE. Has anyone used both Pycharm and VS Code? Any comparison from personal experience?

13

I’ve used both, and both are great. If you’re a serious IDE user, Pycharm does more (eg refactorings)

1 Like
14

If you use VS Code, make sure you install the Python extension, it adds linting, code completion, debugging, and some pre-defined snippets. PyCharm still has more features specific to Python but VS Code is a better editor overall, especially if you do more than just Python. Sublime used to be my goto editor, but it is unofficial abandonware, he still makes updates but he has been promising 3.x release for years and keeps disappearing for 6 months at a time between minor updates.

15

Thanks for sharing @dradientgescent. I have been using sublime but will test out pycharm over the next few weeks.

As a general follow up to the discussion, I brought up using a good text editor with a remote cluster with my engineering team and it actually spurred an interesting discussion. Some engineers think that getting to a level of comfort with vim is the best long term strategy. As someone new to vim who finds it to be a fickle tool I was surprised that people find it usable for extensive changes to code. Of course others totally agreed to set up your system to suit your coding preferences.

It did make me think maybe I haven’t given vim a fair chance. I’m on mobile now, but I’ll share a great resource a colleague sent me about getting comfortable with vim as an editing tool. :slight_smile:

1 Like
16

Maybe this is widely known, but I just found out about mosh, i.e. Mobile shell.

Similar in usage to SSH but much better with weak internet connections, etc. Main downside is no tunneling.

1 Like
17

“Fickle”?.. It’s the most powerful text editor I’ve ever used. It has a steep learning curve, but the payoff is great. To call it “usable” for extensive changes is quite the understatement!

1 Like
18

hahaha thanks @jeremy, I will invest in a better relationship with vim. :slight_smile: this was recommended to me as a great resource: https://danielmiessler.com/study/vim/#gs.null.

19

Thx. I was looking for a working ssh approach to deal with spot instance (changin IPs), will give this a try

20

Agreed.

This is what my vim looks like:

SecureCRT_2017-03-17_19-05-40.png1158×693 30.9 KB