Lean onto the interests of assistant professors at your school instead of trying to find promising RL subtopics.
This way you can get time with someone who’s deep in the field and can be a guiding light for your projects.
If you’re really into embodied AI, you can add that to an existing project but doing it by yourself can be extremely tough especially if your topic requires a lot of compute and your last name is not Rockefeller.
And in general, depending on "funding", simply tapping latent ideas from profs will be an instant win-win. They've wanted to try it, and you've wanted to do work in the field. Approach it that way, as a meaningful collaboration for both of you - and if you have more than one option, then bias towards personality match and "fun" factor.
My advice, as somebody who has a master’s degree, is to pick a topic that you are really interested in, and that captures your imagination. You will have to dedicate yourself to this work for a considerable amount of time.
Doing a CV-driven master thesis will be miserable and more likely to fail. It should be something that is fun and captivating.
Intrinsic motivation/curiosity driven exploration/novelty seeking feels like it has a breakthrough paper waiting. If someone could get those methods working for LLMs, we start getting things like move37 but in math proofs and then everything else
On Policy Self Distillation and Active Learning. Anything that increases sample efficiency by providing a richer more dense feedback signal and is more efficient at exploration / sampling.
ai didnt takeover the world like it was supposed to because its only good at verifiable stuff like coding but vast majority of economy isnt verifiable.
Depends on the structure of your Msc, the funding, the compute available, etc, you need to give those details if you want a detailed answer. If you choose something at the limits of the compute available to you, then you make things 10x as difficult.
The easy answer that applies everywhere is, choose a topic that has a good volume of phds, postdocs, or professors at the college you're joining. The deep knowledge that arises from this no matter which subfield in RL it is in will later help you transition to your desired subfield much better than a surface-level effort directly in your desired subfield.
The most _likely_ answer is sim2real. It has just the right mix of generative AI (funding), robotics usecases (there is an easy "end" for your thesis to hook onto as the "impact"), and lots of unexplored paths (you won't be chasing a common frontier competing with hundred others). But like I said this depends on the previous two points.
Of course there's what you're passionate about and so on, but at the end of the day objectively deep knowledge matters more than passion, unless you're so deep into something, but then your choice is made and you wouldn't have asked this question, so prioritise deep knowledge. Try to actively reduce breadth, IMO the purpose of a research-oriented program is to learn to study depth-wise to the end.
As for RL in BCI, as far as I know, we still use established "RL 101" algorithms directly applied to the brain signals. Any research deep enough (but short-term enough for a MSc) here will be less RL theory/algorithms and more about the details of this specific application. If that's what you want, go for it.
General advice from a CV PhD dropout:
Lean onto the interests of assistant professors at your school instead of trying to find promising RL subtopics.
This way you can get time with someone who’s deep in the field and can be a guiding light for your projects.
If you’re really into embodied AI, you can add that to an existing project but doing it by yourself can be extremely tough especially if your topic requires a lot of compute and your last name is not Rockefeller.
Concur! (CS PhD)
And in general, depending on "funding", simply tapping latent ideas from profs will be an instant win-win. They've wanted to try it, and you've wanted to do work in the field. Approach it that way, as a meaningful collaboration for both of you - and if you have more than one option, then bias towards personality match and "fun" factor.
It is especially important to work with early-stage researchers, as they are more eager to publish and obtain career stability than full professors.
thanks! i found this approach helpful in my undergrad studies as well
My advice, as somebody who has a master’s degree, is to pick a topic that you are really interested in, and that captures your imagination. You will have to dedicate yourself to this work for a considerable amount of time.
Doing a CV-driven master thesis will be miserable and more likely to fail. It should be something that is fun and captivating.
Intrinsic motivation/curiosity driven exploration/novelty seeking feels like it has a breakthrough paper waiting. If someone could get those methods working for LLMs, we start getting things like move37 but in math proofs and then everything else
RL for closed-loop adaptive BCIs and sim-to-real robotics still look like the strongest practical directions right now.
I would probably follow whatever Farama is doing, https://farama.org/projects especially any of the (MO) multi-objective stuff ...
On Policy Self Distillation and Active Learning. Anything that increases sample efficiency by providing a richer more dense feedback signal and is more efficient at exploration / sampling.
things that are hard to verify.
ai didnt takeover the world like it was supposed to because its only good at verifiable stuff like coding but vast majority of economy isnt verifiable.
Depends on the structure of your Msc, the funding, the compute available, etc, you need to give those details if you want a detailed answer. If you choose something at the limits of the compute available to you, then you make things 10x as difficult.
The easy answer that applies everywhere is, choose a topic that has a good volume of phds, postdocs, or professors at the college you're joining. The deep knowledge that arises from this no matter which subfield in RL it is in will later help you transition to your desired subfield much better than a surface-level effort directly in your desired subfield.
The most _likely_ answer is sim2real. It has just the right mix of generative AI (funding), robotics usecases (there is an easy "end" for your thesis to hook onto as the "impact"), and lots of unexplored paths (you won't be chasing a common frontier competing with hundred others). But like I said this depends on the previous two points.
Of course there's what you're passionate about and so on, but at the end of the day objectively deep knowledge matters more than passion, unless you're so deep into something, but then your choice is made and you wouldn't have asked this question, so prioritise deep knowledge. Try to actively reduce breadth, IMO the purpose of a research-oriented program is to learn to study depth-wise to the end.
As for RL in BCI, as far as I know, we still use established "RL 101" algorithms directly applied to the brain signals. Any research deep enough (but short-term enough for a MSc) here will be less RL theory/algorithms and more about the details of this specific application. If that's what you want, go for it.
appreciate the answer!