Here’s some more detail on the vision I sort of have for this thing:
First, I think it would be beneficial to have an in-person “core” component of the group that can meet regularly in the Bay Area. People outside the area would be welcome to participate via Zoom or something like that, but I do think it’s beneficial for a lot of reasons for the people who can get together in person to get together, and we should enable that interaction. Thus, I’m thinking of this as a local group that others can (and should) participate in remotely, rather than a group designed to be done completely remotely. If this got big enough (which would be crazy), maybe there can end up being multiple in-person groups, but I’m certainly not thinking that far ahead haha.
I’m thinking of this more as a “research and development” group than a “study group”. By that I mean the goal in and of itself is not to “learn” the fundamentals of NLP, but to extend the practical range of NLP by both pushing the state of the art and implementing the tools to make those methods available for easy use. In that sense, there would be an assumption that participants would already have some experience with NLP and deep learning along with some coding maturity, or be capable of getting up to speed on their own relatively quickly (which in my opinion wouldn’t be that difficult, but mileage may vary).
My guiding approach is that practical >> cool. I’d like the group to be strongly practically focused on techniques that are viable in the near term rather than the long term (Think company R&D type stuff over academic research type stuff). Which for example means favoring topics like transfer learning and language models and domain-specific stuff, but disfavoring topics like AGI or reinforcement learning or other things that will have little/no utility in the next few years. Not as sexy, sure, but more useful to more people now, with the highest “ROI”.
In that spirit, think of this less as a traditional “seminar” where people get together and somebody gives a talk on a paper and people ask questions about it, and more in identifying, studying, and (most importantly) discussing the practical viability of modern NLP ideas and implementing them if they seem fruitful. These don’t have to be implementing papers, but can come from implementing your own ideas as well.
With this in mind, there would ideally be a development component to this. If we identify stuff that might work well, but isn’t easily available in libraries now (which will usually be the case), we can implement the tools to make those techniques easier to use. How this would be done, of course, would still be a matter up for debate. It would all be open-source, of course.
To make this work, I think it would be beneficial early on to identify research thrusts. For example, some might be specifically interested in NLP applications to medicine, and may want to work on stuff specialized to that (e.g. medical-specific language models). Others (including myself) might be interested in extending the utility of transfer learning to as many useful NLP tasks as possible, and can experiment with and hopefully implement the tools to do that. Others might be interested in improving language models, or chatbots, etc. Of course, people can be involved in multiple thrusts if it suits them, but the goal would be active over passive participation.
Here’s sort of how I’d think of a typical meeting. At the beginning, each thrust area would share with everyone what they’re working on and what cool stuff they’ve found, which can result in some discussion. After that, each thrust would get together and work on their own defined goals. They’d have put together a reading list (like over Slack) and already have mostly read any “papers of the day”. Each person in the thrust would’ve decided on something they were going to try (e.g. reproducing a paper discussed before, running new experiments, or implementing software tools), and the bulk of the rest of the meeting would be a kind of group coding thing where people can collaborate and stuff like that.
I still want to flush the details out more, and welcome feedback, but these are at least at a high level what I intended the “founding goals” of this group to be. It’s obviously not for the feint of heart and may be a lot more than some of you were expecting or hoping for,. It would require more work on everyone’s part. But I think the outcome would be a lot more rewarding for everyone in the long-term by going this route, as opposed to some general “paper discussion” group or beginner’s “study group” or something like that.
That being said, if this doesn’t scare you off, I’ll be glad to work with you to set something up and hopefully get something going in the next couple weeks. Now that I think about it, I shouldn’t have limited this post to the private “Part 2 (2019)” group, but oh well. I’ll figure out how to fix that later.