You can find last year’s blogpost here.
The essay begins with the first step: deciding what to work on. Thankfully, as a graduate student, this task is mostly left to the advisor. In my case, I was very fortunate to be advised by Stephen Boyd, who wanted someone to extend CVXPY to interface nonlinear programming (NLP) solvers.
What are you excessively curious about — curious to a degree that would bore most other people? That’s what you’re looking for.
Although the project was proposed by Stephen, I’d like to take a bit of credit for being "exceedingly curious" in the first place. Having contributed to CVXPY for almost two years prior to the visit, I was very invested in this idea of working on an extension for NLP, probably more so than most people. In addition, after some discussions with Steven Diamond, I learned that this project was in demand for a long time, both from Stephen and other members of the community, which further motivated me to have another attempt at getting it done.
During the rest of the essay, Paul presents different "themes" that relate to doing great work. I have selected a few which I think are worth highlighting.
The first thing that resonated with me was Paul’s emphasis on working on unfashionable and overlooked problems.
[…] the most common type of overlooked problem is not explicitly unfashionable in the sense of being out of fashion. It just doesn’t seem to matter as much as it actually does.
He also adds that:
One of the most interesting kinds of unfashionable problem is the problem that people think has been fully explored, but hasn’t.
I think this was the case for DNLP. Interfaces to NLP solvers have existed for decades, most people, including me, probably thought there wasn’t anything interesting to explore. In fact, our project isn’t even the first NLP interface written in Python, as Pyomo and PyOptInterface already existed for a long time. The project is also fundamentally different than fashionable projects of our time: GPU acceleration, differentiable optimization and AI.
Great work often takes something that already exists and shows its latent potential.
The difficulty of using non-differentiable functions with NLP solvers has created many interesting alternatives: some use boolean variables, others just ignore them completely, and the rest mostly restrain from using them in the first place. With DNLP, we were able to explore the potential of a fourth option: a discipline that guarantees a lossless transformation to an equivalent smooth program.
Paul then mentions outlier ideas as another possible factor for great work.
Boldly chase outlier ideas, even if other people aren’t interested in them — in fact, especially if they aren’t.
I don’t think by any means DNLP was an outlier idea, but our first attempts at it could certainly fit in that category. We tried many different reformulations which violated all sorts of constraint qualifications. I think most people in the field, especially the experts, would have been completely disinterested in the things we tried initially. Paul adds that:
If you’re excited about some possibility that everyone else ignores, and you have enough expertise to say precisely what they’re all overlooking, that’s as good a bet as you’ll find.
And again, a lot of credit must go to Stephen; he was probably excited about this possibility for a couple of years by now. I remember asking him early on about the motivation to do this project and he simply said: "it should have been done a long time ago". Although, as the project shaped itself and our expertise developed, we were all definitely more aware of this possibility as well.
The next component of great work is something that’s obvious: working hard and putting in the hours.
Great work usually entails spending what would seem to most people an unreasonable amount of time on a problem.
I recall one critical point of our project which was about computing derivatives.
In an initial prototype, we used cvxtorch which converts CVXPY expressions to PyTorch ones.
Then derivatives could be computed using the convenient backward() method.
However, during a meeting, Stephen pointed out that it wasn’t necessary, with a clever reformulation of the problem, we could guarantee that derivatives could all be computed analytically.
This took us a tremendous amount of work, basically requiring us to write the Jacobian and Hessian of every atom from scratch, verify the canonicalization of all atoms, and write many many tests.
But we were absolutely happy to go through this process if it would help the final product (and it most certainly did.. in hindsight, maybe it was one of the best decisions we could have made).
Great work happens by focusing consistently on something you’re genuinely interested in.
I think the combination of positive feedback from Stephen, and the software improving day by day, allowed us to stay genuinely interested in the project for a very long time.
When you pause to take stock, you’re surprised how far you’ve come.
And it turns out, after months of making some small progress daily, we had test coverage for all important atoms, and also dozens of examples in various applications to showcase the usability of the new NLP interface.
Then Paul begins to talk about assessing the quality of your ideas.
When an idea seems simultaneously novel and obvious, it’s probably a good one.
I felt something similar on the day we formalized the grammar behind DNLP. At the time, it seemed like such a simple and elegant idea, and yet, it took us many different versions before getting it right.
That’s how new ideas can be both obvious and yet hard to discover: they’re easy to see after you do something hard.
At the beginning we were slightly worried that people would comment on DNLP saying it is "simply" replacing affine with smooth in the DCP rules”. I am now confident to reply that "yes, it literally is", because in hindsight it is truly just that simple.
Finally, Paul highlights a critical aspect of doing great work:
Seek out the best colleagues.
I didn’t explicitly do this, as I was focused on getting things working in the beginning. But through some sheer luck it kind of happened naturally. I met Daniel Cederberg as he was teaching 364a for the summer quarter, and after a few discussions, he joined our project. It is without exaggeration that I say meeting and working with him is the best thing that could have happened to me during the visit.
There are a lot of projects that can’t be done alone, and even if you’re working on one that can be, it’s good to have other people to encourage you and to bounce ideas off.
This point was also particularly relatable for me. Through the countless implementations of derivatives and canonicalization of atoms, it was absolutely crucial to have someone to debug issues and share ideas with. Not only was Daniel extremely productive and efficient, he has great ideas and an even greater sense of humor. It’s been a tremendous pleasure to work (and continue working) with him.
Colleagues don’t just affect your work, though; they also affect you.
I also got a chance to visit Daniel and his girlfriend Clara in Sweden for almost two weeks. I got to see the many aspects of Daniel’s life back home, and it was very exciting to meet all his family and friends over there.
So work with people you want to become like, because you will.
Maybe that was the reason I enjoyed working with Daniel so much: in some way, I probably wanted to become more like him.
Overall, I think we did truly great work for DNLP. I hope we will be able to explore this new discipline to the fullest extent over the upcoming years. I personally believe that many more fields will benefit from this work and I am greatly excited to see how it will turn out.
It was also very helpful to write this blogpost. Paul’s blogs in general are absolutely insightful, but "how to do great work" is an all-time classic; I definitely recommend you to have a read. I will certainly look at it many more times in the future.
Naturally, some parts of this reflection will sound presumptuous. But I think this is the price that must be paid in order to have such an introspection. Apologies if it irritates anyone. (Although, I doubt anyone truly reads these blogposts, except maybe myself in a few years)
]]>Although I am definitely enjoying the freedom of having less stressful deadlines, I believe I learned a lot from the classes I took during my Master’s (Multivariable systems in Fall 2024, the other three in Winter 2025). I am writing this blogpost as a way to revisit important things I learned in the past two semesters and also share my thoughts on the classes.
Before we begin, I wanted to say a quick word about the line of thought behind the class choices. My supervisor recommended many “modern” classes on theoretical machine learning or reinforcement learning. I definitely was interested in those topics, but felt more inclined towards optimization related classes because there was more to be gained (given my background and interests).
This class is taught by a new professor in the Department of Electrical Engineering, Bowen Yi.
As the name implies, the class is about Multivariable Linear Dynamical Systems. We first talked about modeling and the classic control loop defined with the input, output and internal state variables. The next lecture talked about the discrete linear-time invariant state-space model and its analytic solution using the state-transition matrix. We conclude the first section by learning about the Laplace transform and the Jordan Canonical form to compute the exponential of a matrix (which appears in the analytic solution).
The next part of the course was my favourite, as we began introducing key concepts from control theory: controllability, stability and observability. We first defined what these concepts meant in a mathematical sense for different types of control systems (time variant versus invariant). This required us to learn about the respective gramians and theorems on determining if a system is controllable (observable) or not. To conclude this part of the section, we talked about a very beautiful duality result between the controllability of a system and the observability of its adjoint. I hope to gain a better understanding of this result at some point.
Finally, we talked about the internal state of a control system, more specifically: feedback and observation. This led us to discuss about ways to design a state feedback controller such that the overall system has desired properties (i.e. closed-loop stability). We also talked about canonical forms for controllable systems which was interesting because the same term is used in optimization in a completely different context. For the state observer, a discussion about decompositions of observable and unobservables states led us to the famous Kalman filter (observer). I am certain that I will revisit this one day since I didn’t completely grasp it.
We also had bonus material on realization theory which seems to be a sort of inverse control problem where the goal is to find the control matrices given a proper transfer function (only works for linear time-invariant systems). There were some cool theorems on existence and minimal realizations but I already mostly forgot about them.
Overall this course was truly interesting and allowed me to gain a very very basic understanding of control systems. There are so many cool extensions of this class that I can now explore one day: stochastic control, optimal control, adaptive control, etc. (most are taught at Polytechnique also!). I want to conclude this section for sincerely thanking Professor Bowen Yi, who took the time to explain basic concepts to me and also for having really interesting conversations about the class and existing connections to optimization.
I took this class because there was a conflict with getting credited for taking the class Convex Optimization at McGill as an undergrad. In summary, Polytechnique doesn’t allow students from other universities to have a class credited if that class counted towards their own degree. Interesting rules… but it was a bit out of my control, and I was open on learning new material anyways, so here we are.
Last summer, as I was having a random discussion with Parth, he mentioned the importance of understanding matrix computations for optimization. This was later re-iterated by Sarah (the author of faer.rs), an expert numerical algebraist.
I thus decided to enroll in this seemingly completely unrelated class and must admit that I didn’t attend the in-person lectures much. However, I read most of the lecture notes by myself and also completed all the assignments with the occasional help of one classmate (and a lot of help from copilot).
The class covered many essential topics in numerical linear algebra and practical implementations of algorithms. We went through matrix norms, perturbation and stability analysis, cholesky and $LU$ factorization, blocking algorithms (three levels of BLAS), least squares, $QR$ factorization, singular value decomposition (SVD) and finally, some eigenvalue problems.
There was a lot of material and I definitely didn’t fully understand everything, in my opinion, the last two chapters on SVD and eigenvalue problems were particularly not too well understood. However, I feel like I gained a great appreciation of this field and its importance. I also noticed that many factorizations and algorithms would reappear throughout the class. For example, the SVD and least-squares problem can often be solved with a $QR$ factorization (it is also numerically stable!).
After taking the class, I read some paper on large-scale convex optimization and it was mentioned that the main bottleneck of second-order method came down to efficiently computing the cholesky factorization of the positive-semidefinite matrix that appears in these methods. It was fascinating to me that despite known (and simple) algorithms existing for these problems for more than 100 years, there is still a lot of work to do in practical manners (ex: exploiting sparsity, leveraging hardware, etc.).
I didn’t do so well on the final exam (which was worth 50%) and ended the class with a B. However, I think it doesn’t really matter since, out of all my classes, I am likely going to revisit the lecture notes at some point in my optimization and programming career. (although I am probably saying that to try and comfort myself)
This class was taught by a relatively new professor at the Computer Science department of Universite de Montreal, Kimberly Yu.
I really enjoyed taking this class as we were a small group; professor Yu was quite involved and seemed to care about our learning experience.
We began by talking about mixed-integer formulations for certain problems involving graphs and piece-wise linear functions. Note: this class only covered mixed-integer (linear) formulations. It was interesting to learn some modeling tricks involving binary variables and how it affects the underlying algorithm.
This led us to talk about strength of a formulation, how we should talk about comparing two formulations, etc. It turns out that the best formulation is the convex hull of the integer feasible points, since we can solve the relaxed problem (eliminating binary constraints) and get the optimal integer solution.
We then continued with cover inequalities, a technique to generate good cuts which help the mixed-integer algorithm eliminate bad integer solutions. We also delved into some theory about a different viewpoint of a convex hull which involved extreme points and rays. This alternative representation allows us to construct a so-called extended formulation of the original set. My intuitive high-level understanding of this extended formulation is a sort-of lifting in higher dimensions which describes the convex hull of the original formulation.
We concluded the class by learning a classic cutting method from Chvátal and Gomory, the branch-and-bound method, and also preparing a final presentation on anything that’s related to integer optimization. I decided to do mine on the unit commitment problem since I had some prior interest through working at Gridmatic. It was a cool experience as I learned a lot more details about the problem, and how to clearly explain it. Unfortunately, I was a bit too lazy to implement the results in a real large network.
Our last homework was to do some custom callback implementation in Gurobi, but I mostly used Copilot and the help of another classmate to complete the exercises. I don’t feel confident to say I have learned anything from doing this (other than knowing what a callback in a solver does!).
This class is taught by my supervisor Antoine Lesage-Landry, which used the book: Convex Optimization of Power Systems by Joshua Taylor.
I had quite mixed feelings about the class. On one hand, optimization is my favorite topic and I was super eager to learn about its applications in power systems. On the other hand, I found out that the methods presented are not quite applicable to the operation of real networks and they were extremely sophisticated.
It didn’t help that the pace of the class was absolutely brutal. We would cover 3 different topics in one hour chunks every week, from chance-constrained optimization to SDP and SOCP relaxations of the AC-OPF to radial power networks. We even had a part of the class which talked about reinforcement learning algorithms such as Q-learning and gradient policy learning. I can safely say that I don’t remember much of the content.
On the bright side, I learned a lot through doing the homeworks (which were using CVXPY!), and collaborating with classmates from my lab. We also had a final project, where I did a presentation on differentiable optimization and its applications in power systems. However, I was once again too lazy to do any real implementation and didn’t end up with a good grade. My hope was to learn more about this hot topic and maybe it could help me for the VSR stay with Parth and Boyd’s group (things turned out rather different though).
Note: this paragraph and the content of two classes were added on a much later date (October 31, 2025).
Looking back, I am glad I decided to take classes in 4 completely different topics. It allowed me to learn a lot of new things and, although they are not directly helping my research, I am sure it has helped me develop mathematical skills.
For the time being, I will probably not be doing anymore classes since there are lots of open source and research tasks to do (and I need to write a thesis to graduate, at some point). However, if I ever end up doing a PhD, I would love to take more classes, maybe some stuff in statistics, optimal control and applications, and more CS stuff like advanced compilers and maybe even generative models?.
]]>You can find last year’s blogpost here.
Looking back, this year was pretty eventful.
I first completed my undergraduate degree with four very difficult classes (especially the compiler one). I talk a bit about my undergrad experience and give some very general advice in this blogpost.
I then had some fun moments with my good friends from Concordia. We first all celebrated graduation together, followed by an amazing two week trip to Guatemala. So much happened in that trip that I will remember for a long time, we climbed the Acatenango volcano, made friends with other tourists from all across the world, and took boat rides across Lake Atitlan. I want to thank Tommy Hegarty for making this trip possible and allowing the rest of us to discover this beautiful place.
I then flew to California to work for a summer at Gridmatic and do more open source contributions for CVXPY. I first settled in East Palo Alto and was doing a terribly long biking commute to the office in Cupertino. But luckily my coworker David Pineiro was leaving the country for a small period of time for his mandated vacation (O1 visa) and I got a chance to take over his room in Palo Alto (on the legendary Los Robles avenue). I also got a chance to meet other fellow CVXPY maintainers in Parth, Philipp and Steven and learned a lot from working in-person with them. I talk a bit more about my summer and what followed in this blogpost.
Another highlight of the year was attending the ISMP conference in July which was located in Montreal. I was very fortunate to meet Bartolomeo Stellato, a Princeton professor and one of the authors of OSQP. I even got a chance to grab dinner with his incredible PhD students (Irina, Rajiv and Vinit) as well as some of his collaborators.
Finally, I came back to Montreal to start my Master’s degree at Polytechnique. I restarted playing hockey once a week with my labmates Xavier and Étienne which has been a real pleasure. I also met some really fantastic people working in all kinds of energy applications from contingency screening, control techniques in virtual power plants and HVAC management for buildings. I had some struggles in my control theory class, mostly due to lack of focus, but was able to redeem myself in the final exam.
For the second year in a row, I have accomplished none of my yearly resolutions.
The Rust Backend project for CVXPY was probably way too ambitious; I didn’t even write a single line of Rust this year. This is not to say that the year was a failure but rather, that I had other higher priority tasks to take care of first. When there was a bit of free time for the backend, I would naturally choose to rest and do other things. My feeling is that the project is a bit too daunting at the moment and that it needs to be split up into more manageable chunks, certainly to be continued.
My second goal was to contribute to a journal paper and I am also quite far from achieving that. My first Master’s semester at Polytechnique was overall pretty good, I met some incredible people in the lab, read quite a few papers and most importantly, learned a lot of things related to optimization and its applications in power systems. Although we have dropped the idea of implementing Input Convex Neural Networks from last year, we have now many more avenues to explore. I will name a few of these ideas for a future version of myself: Stochastic Newton’s Method with zeroth order oracles, constrained SOONO, and gauge projections for fast online optimization.
I truly had expectations that were far beyond my abilities at the beginning of this year, but at the same time, I was also able to accomplish other things that were completely unimaginable for my past self. However, more importantly, throughout the year, I have gained a deep sense of fulfillment while sharing/exploring research ideas with others and helping CVXPY users through a lot more open source development.
For the winter semester, I am enrolled in three classes to complete the Master’s credit requirement (apart from the thesis): Integer Programming, Optimization and machine learning for power systems and Matrix Computations. My first goal for the year is to do well in these classes and broaden my understanding of different aspects of optimization. An important project of mine is to add more examples to CVXPY’s library inspired by the exercises in these classes, potentially making use of sphinx-gallery to improve the look of the page as well.
My second goal will be to get a clearer idea of what my thesis will constitute of. After the calendar year 2025, I will only have two more semesters to complete my thesis, and potentially one of those semesters will be used to do a research stay abroad (updates in another blogpost). I hope that despite the exciting things that I look forward to this year, in particular traveling in Turkey and/or China, the ICCOPT conference, and maybe visiting some friends at Gridmatic, I will be able to dedicate quality time towards progress in the research project.
Here’s to another great year of learning and hard work!
]]>After my summer project for Google Summer of Code (GSoC), I expressed my interest to continue making contributions to the CVXPY team. I thus got the opportunity to work on improving CVXPY’s webpage and documentation, more details about this small project can be found here. While working on this project, I remained actively interested in the other issues that popped up on GitHub and made several other unrelated contributions.
Fast forward a few months, I somehow got an opportunity of a lifetime to do an internship at a startup in California. After obtaining a work visa (J1) and getting the other paperwork sorted out, I moved to Palo Alto for three months.
I will keep this section short because it is only slightly relevant to the post.
The startup in question is called Gridmatic and it uses optimization and machine learning to operate batteries and participate in convergence bidding. I was extremely fortunate to not only continue contributing to CVXPY, but was actually encouraged to tackle long-lasting open issues which would see internal usage at the company. Many contributions in the 1.6 release were effectively funded by Gridmatic, and I am very thankful for that.
As an ode to the original GSoC, I am now calling this summer of open-source contributions: Gridmatic Summer of Code. I also wanted to quickly thank the amazing researchers and engineers at Gridmatic for an unforgettable summer filled with interesting discussions and fun social activities.
I will now talk a bit more about my contributions in the release.
Although I have made a lot of PRs in the past few months (as evidenced by the release notes), the bulk of my work can be summarized in three main features: N-dimensional expressions, sparsity attribute, and other performance improvements. The rest of the pull requests mostly involved fixing/improving the CI, documentation and adding/dropping support for Python versions.
I was going to go a bit more in detail about these new features, but I feel like they could each warrant a deepdive blogpost of their own (not to mention I have been somewhat lazy in terms of writing). Instead, I included hyperlinks to the pull requests in question above.
For my own future reference, I will still say a quick word on these new features. Both the N-dimensional exressions and sparsity attribute have similar flavors, they greatly improve the representation of optimization problems and can also improve their performance (solve and canonicalization time). Due to the symbolic representation of expressions in CVXPY’s backend, the implementation of these two features involves coming up with a sparse coefficient matrix that performs the desired operation on any general expression.
The difficulty mostly lies in being able to cover all types of inputs, shapes and other given function data (i.e. axis, keepdims, parameters, etc). I have to give a lot of credit to people from the CVXPY team whom were often the ones figuring out the clever details. I was mostly there to implement, test and validate if their ideas would turn out correct or not.
One of my happiest moments this release was seeing new contributors tackling important missing features. In this section I wanted to give a special shout-out to the following contributions as well as the issue they resolved:
If you look at those contributions you will notice that I did code reviews on all of them. Even though I wasn’t familiar with parts of the codebase that was affected, I wanted to make sure that the changes were adequate and that I somewhat understood them. While doing these reviews, I remarked that I had gained some shared responbility; I would be (atleast partially) held accountable if things went awry because of the newly merged pull request. This reinforced in me the saying that writing code is not the only way to contribute to open-source.
I will end this section with a quote I first heard from Steven that I wholly believe in, it goes something like: “what’s always really important in open source are not the users but actually the contributors”.
Although I recently took a break from writing code to focus on my master’s, I am still looking forward to contribute once again.
The following things are on our agenda:
I am also taking a graduate class on optimization and machine learning in power systems next semester and am hoping to add some (or write my own) examples to the library.
To end this blogpost, I just wanted to say thank you to the amazing CVXPY team, namely Steven, Parth, Philipp and Riley. I have learned a tremendous amount through your code reviews, discussions and mentorship. In the off-chance that one of you read this, please know you have inspired me a lot and it is an absolute honor to work with you.
]]>Due to the difference in high school curriculums between Quebec and Ontario, out-of-province students need to take an extra year of university classes which is formally known as the extended credit program (ECP).
There is a recommended list of elective classes to take to satisfy those credits, which cover various topics like economics, philosophy and astronomy. However, not many people know that you are actually able to take almost any class and have it satisfy your ECP electives.
Due to my interest in music, I got a chance to take a jazz theory class which was really an awesome experience. I also used another elective slot to take an additional math class that wasn’t part of my main curriculum.
If you ever thought of taking a very random class out of pure curiosity, this is your best chance, but please make sure to ask the registrar before enrolling to make sure the transfer will be possible.
There is a portal that allows Quebec university students to take classes at any other institution within the network. Through this opportunity, I got a chance to take an advanced math class (Convex Optimization with the excellent professor Courtney Paquette) at McGill university.
This portal is great because if some classes are only taught on certain semesters at your university, you might be able to take an equivalent class elsewhere at your own pace. Of course, the main benefit is that you will get to experience another student life and meet new and interesting people.
Another little trick that not many people know about; You can actually skip pre-requisites at the host university, as long as you are able to transfer the class to an equivalent one at your home university.
For example, the optimization class at McGill focused a lot on theory, analysis was a pre-requisite, but since Concordia didn’t have the same requirement for its optimization class, I was effectively able to bypass this rule. This is only something I would recommend doing if you are absolutely interested in the class and do your own research beforehand.
This is sort of related to the previous advice, but might be a bit more challenging both financially and logistically. The best time to do this exchange would be on either your 2nd or 3rd year. I already talk about my experience in this blogpost, but to summarize, doing a semester abroad is a great chance to travel around the world, learn about a new culture, and meet new friends.
In my last year, because I was in the honours program, I had to complete a project under the supervision of a professor. It was a great experience as I developed some basic research skills and was able to manage my time without strict deadlines. I proposed a project, which combined my programming background with the mathematical knowledge of my supervisor, that eventually led to an open source contribution to NetworkX. In case you are curious to know more, you can read my final project report here.
Doing this project at a relatively early stage of my student career gave me the confidence that I could continue doing research in graduate school. Even if you are not considering further studies, doing such a project can be a truly rewarding experience. It will allow you to explore ideas, read journal papers, work with incredible researchers, and potentially make a contribution to science.
In my opinion, the most valuable thing I retained from my time as an undergraduate student (obviously, apart from the amazing people I have met) are the skills I gained from internships and developing open source software.
For anyone who doesn’t have summer plans, I would really recommend to try and find internships or any other opportunity to develop your skills. Although it can be really stressful and competitive, the things you can learn are truly invaluable. You will have a chance to work on large codebases with real production impact, learn programming best practices, and most importantly, develop communication skills.
At the end of the day, programming is about working with people, understanding what others have done and being able to slowly form your own feature ideas as well. There is no better way to improve than practicing working on large projects with other experienced developers.
Finally, I would like to end this blogpost by saying these are just some things I noticed had a profound impact on where I find myself today, you might have completely different goals from your undergraduate education and that is completely fine. I noticed that we naturally tend to things we like, so just go with the flow.
]]>You can find last year’s blogpost here.
The first few months of the year I went on an exchange at the University of Manchester.
I learned to live and travel independently and developed some impactful friendships. It definitely still stands as one of my most memorable experiences to date.
You can read more about my exchange here.
During the summer, I participated in the Google Summer of Code program with CVXPY.
I ended up making some important contributions that brought major performance improvements to the canonicalization process.
I also had the chance to meet with my GSoC mentors Philipp and Steven during the SciPy 2023 conference in Austin, Texas.
You can read more details about my brief summer update here.
Finally, I came back to Concordia to complete my second to last semester of undergrad. I completed my undergraduate project along with four other classes. I also applied for graduate school and plan on detailing my journey in a future blogpost.
If you’ve read my review from last year, you might wonder how many of my learning resolutions did I manage to tackle?
And the answer would be absolutely none.
But do not despair, as I am almost certain I will finally be able to take on two of last year’s resolutions: learning Rust and reading journal papers.
With a continued interest in working on Open Source and improving canonicalization performance, a CVXPY Rust backend is at the top of my priority list.
In addition, having made contact with a Professor from Polytechnique Montréal, I was already assigned a few preliminary papers to read on the topic of Input Convex Neural Networks. Albeit, my goal isn’t to fully understand the theory but instead being able to apply some new techniques to power system applications.
As for other learning goals, given that I am about to take a Compiler Design class, I would be excited to get a deeper practical understanding of topics in parsing and compilers.
I believe the skills and knowledge I acquire in this class could potentially translate to future improvements in CVXPY and elsewhere.
Finally, I would like to set a harder goal of contributing in a published journal paper. I think with enough preparation and luck, some opportunities could turn out to be fruitful.
]]>Today I had an assignment for my writing class, the goal was to write a critical review on a journal paper, article or book. I decided to choose a topic that is deeply relevant to our time: usage of AI tools in computing education. I think I did a decent job, so I decided to share what I wrote.
This critical review is on a journal paper [LG23] written by Sam Lau and Philip Guo, two researchers in the department of NeuroScience at the University of California, San Diego. The paper talks about the evolving landscape of education in the age of AI code generation and large-language models (LLMs). Through questioning a wide variety of programming instructors, the authors attempt to give an overview of the short-term and long-term plan of educators in regards to the rapid rise of AI generation tools. I personally found this paper really informative as it explains creative solutions to deal with the evident problems of cheating and lack of understanding the material from the students.
The paper first gives an overview of many possible ways that software developers and researchers have been using AI code generation tools. The most notable ones are code completion, code simplification, debugging support and conversational bug finding. However, the author also states some critical limitations to these AI generation tools such as inaccuracies and low code quality. In a related article [Guo23], Professor Guo gives an overview of other ways scientists can incorporate AI tools into their existing workflow to enhance productivity and learning. This begs the question, if the usage of AI tools is accepted and even promoted at the workplace, shouldn’t educators also try to teach their students how to make proper usage of upcoming AI technology?
The main findings of the paper consist of the summary from short interviews with 20 instructors of introductory Computer Science classes across the world. To begin with, most professors mention that it was difficult to know how much their students were using AI tools with one of the TAs stating that it almost felt like a “taboo topic” [LG23]. In fact, if a student was using these tools, there was no incentive to tell the professors about it. In contrast, all the educators mention that they had previously heard or discussed with colleagues on this subject matter. This ranges from casual discussions, to long email threads, and in the case of one educator, even the issual of a new policy banning the use of AI tools in all classes across the university.
Naturally, as anyone could imagine, all educators were concerned about cheating concerns in the short-term. This led to different types of changes to the structure of their courses; some professors decided to increase the weight of in-person written exams while others tried to ban AI tools in the classroom (as is the case with this current class). In all cases, the participating teachers noted that these were only temporary solutions and that stronger policies and changes needed to be enforced in the future. Here is where things get interesting, participants had vastly different opinions on the longer-term solutions: some proposed ideas to resist AI tools while others wanted to embrace their usability. The former preached the importance of learning the fundamentals of programming while the latter insisted that AI tools could enable students to focus on the design aspect without needing to battle with syntax and rote memorization.
As the author points out, there are many limiting factors to the findings of this journal paper. Despite being affiliated with universities spanning all continents, the classes were all taught in English and the teachers were also all originally from American universities. Another important factor to consider is the type of classes was limited to introductory university courses, thus it might not be generalizable to other levels of education. However, this journal does provide an interesting perspective on computing educator’s reaction to the beginning of worldwide adoption of AI tools. The structure of the paper, from introducing usages of AI tools, to detailing short and long-term solutions, and finally concluding with related open research questions, really helps understand the overall idea the authors were trying to convey. I think what this paper does especially well, is present results objectively, always acknowledging different perspectives, while also relating these concerns to other fields such as equity, access and pedagogy.
To conclude, this paper gives an excellent overview of the usage of AI tools in education from the perspective of university educators. It presents several ideas that are intuitive to the reader along with many surprising facts about the topic. However, this paper raises a lot more questions than it answers. For example, how does the usage of these AI tools affect software developers and scientists across other industries? What kind of repercussions can decisions in education have on the future landscape of jobs and opportunities? All in all, understanding different perspectives on the usage of new information communications technology (ICT) is a crucial step towards forming your own opinion in the subject matter. It will be interesting to see, in hindsight, how accurately the educators predicted the future of education, but alas, as with many things, only time can tell.
[Guo23] Philip J. Guo. Six opportunities for scientists and engineers to learn programming using ai tools such as chatgpt. Scholar articles, August 2023.
[LG23] Sam Lau and Philip J. Guo. From “ban it till we understand it” to “resistance is futile”: How university programming instructors plan to adapt as more students use ai code generation and explanation tools such as chatgpt and github copilot. In Proceedings of ICER 2023: ACM Conference on International Computing Education Research, Aug 2023.
]]>A few days ago, I submitted my final report for the Google Summer of Code program.
It marks a big milestone in my life, and perhaps my proudest achievement to date, but it is also only the beginning of many more interesting things to come.
The core contribution of the GSoC project, the stacked-slices backend PR, is currently ongoing its final review stages.
My mentor and I have added many more backend unit-tests and further detailed some complex operations in the docstrings.
The next task will be to implement a backend using python-graphBLAS. This idea was initiated by meeting some wonderful people at the SciPy 2023 conference.
Overall, the tensor operations will remain intact, we simply need to change the sparse data type and make a few adjustments. I believe this backend could see major improvements due to how it optimizes sparse operations with delayed objects.
For my undergraduate project, I decided to work with my Combinatorics & Graph Theory teacher on exploring implementations of his research in Open Source. You can find a copy of my proposal here.
The main objective is to define a set of heuristics for broadcasting in general graphs and add these implementations to NetworkX’s graph algorithm suite. I will be collaborating with Cédric Paradis, a good friend of mine who is going to work at Morgan Stanley. If time allows it, we will also add comprehensive notebooks on applications of broadcasting and give examples of how to use the new APIs.
Surprisingly, this idea was also inspired by attending a talk at the SciPy 2023 conference. Perhaps attending conferences is worth it after all… (this one definitely was!)
In fact, the people from NetworkX are actually working closely with the core developers of python-graphBLAS to integrate a new “official” backend for graphs in Python. You can read an excellent summary of this partnership here.
In addition to the never-ending Open Source development and the undergraduate project, I will also be completing four other courses at Concordia. Here is the list:
I am clearly most excited about Abstract Algebra since I get a chance to formalize a lot of concepts I am sort of familiar with such as: groups, rings and homomorphism.
In Linear Algebra 2, I will get a chance to solidify some concepts I have learned in the previous version of the course. In addition, many concepts in the outline seem to be highly relevant to optimization theory. It never hurts to learn more math, as they say (I don’t think anyone actually said that).
As for the last two courses, I can’t lie, I wouldn’t have taken them if they were not mandatory. That’s all I will say about them.
]]>