Yeah fake. No way you can get 90%+ using chatGPT without understanding code. LLMs barf out so much nonsense when it comes to code. You have to correct it frequently to make it spit out working code.
If we’re talking about freshman CS 101, where every assignment is the same year-over-year and it’s all machine graded, yes, 90% is definitely possible because an LLM can essentially act as a database of all problems and all solutions. A grad student TA can probably see through his “explanations”, but they’re probably tired from their endless stack of work, so why bother?
If we’re talking about a 400 level CS class, this kid’s screwed and even someone who’s mastered the fundamentals will struggle through advanced algorithms and reconciling math ideas with hands-on-keyboard software.
- Ask ChatGPT for a solution.
- Try to run the solution. It doesn’t work.
- Post the solution online as something you wrote all on your own, and ask people what’s wrong with it.
- Copy-paste the fixed-by-actual-human solution from the replies.
Two words: partial credit.
deepseek rnows solid, autoapprove works sometimes lol
deleted by creator
You mean o3 mini? Wasn’t it on the level of o1, just much faster and cheaper? I noticed no increase in code quality, perhaps even a decrease. For example it does not remember things far more often, like variables that have a different name. It also easily ignores a bunch of my very specific and enumerated requests.
deleted by creator
Asking it to write a program that already exists in it’s entirety with source code publicly posted, and having that work is not impressive.
That’s just copy pasting
deleted by creator
These things work by remind how likely other words are to appear next to certain words. Do you know how many tutorials on how to code those exact rules it must have scanned?
deleted by creator
o3 yes perhaps, we will see then. Would be amazing.
Are you guys just generating insanely difficult code? I feel like 90% of all my code generation with o1 works first time? And if it doesn’t, I just let GPT know and it fixes it right then and there?
deleted by creator
Garbage for me too except for basic beginners questions
I just generated an entire angular component (table with filters, data services, using in house software patterns and components, based off of existing work) using copilot for work yesterday. It didn’t work at first, but I’m a good enough software engineer that I iterated on the issues, discarding bad edits and referencing specific examples from the extant codebase and got copilot to fix it. 3-4 days of work (if you were already familiar with the existing way of doing things) done in about 3-4 hours. But if you didn’t know what was going on and how to fix it you’d end up with an unmaintainable non functional mess, full of bugs we have specific fixes in place to avoid but copilot doesn’t care about because it doesn’t have an idea of how software actually works, just what it should look like. So for anything novel or complex you have to feed it an example, then verify it didn’t skip steps or forget to include something it didn’t understand/predict, or make up a library/function call. So you have to know enough about the software you’re making to point that stuff out, because just feeding whatever error pops out of your compiler may get you to working code, but it won’t ensure quality code, maintainability, or intelligibility.
My first attempt at coding with chatGPT was asking about saving information to a file with python. I wanted to know what libraries were available and the syntax to use them.
It gave me a three page write up about how to write a library myself, in python. Only it had an error on damn near every line, so I still had to go Google the actual libraries and their syntax and slosh through documentation
the problem is more complex than initially thought, for a few reasons.
One, the user is not very good at prompting, and will often fight with the prompt to get what they want.
Two, often times the user has a very specific vision in mind, which the AI obviously doesn’t know, so the user ends up fighting that.
Three, the AI is not omnisicient, and just fucks shit up, makes goofy mistakes sometimes. Version assumptions, code compat errors, just weird implementations of shit, the kind of stuff you would expect AI to do that’s going to make it harder to manage code after the fact.
unless you’re using AI strictly to write isolated scripts in one particular language, ai is going to fight you at least some of the time.
I asked an LLM to generate tests for a 10 line function with two arguments, no if branches, and only one library function call. It’s just a for loop and some math. Somehow it invented arguments, and the ones that actually ran didn’t even pass. It made like 5 test functions, spat out paragraphs explaining nonsense, and it still didn’t work.
This was one of the smaller deepseek models, so perhaps a fancier model would do better.
I’m still messing with it, so maybe I’ll find some tasks it’s good at.
from what i understand the “preview” models are quite handicapped, usually the benchmark is the full fat model for that reason. the recent openAI one (they have stupid names idk what is what anymore) had a similar problem.
If it’s not a preview model, it’s possible a bigger model would help, but usually prompt engineering is going to be more useful. AI is really quick to get confused sometimes.
It might be, idk, my coworker set it up. It’s definitely a distilled model though. I did hope it would do a better job on such a small input though.
the distilled models are a little goofier, it’s possible that might influence it, since they tend to behave weirdly sometimes, but it depends on the model and the application.
AI is still fairly goofy unfortunately, it’ll take time for it to become omniscient.
A lot of people assume their not knowing how to prompt is a failure of the AI. Or they tried it years ago, and assume it’s still as bad as it was.
They’re clever. Cheaters, uh, find a way.
isn’t it kinda dumb to have coding exams that aren’t open book? if you don’t understand the material, on a well-designed test you’ll run out of time even with access to the entire internet
when in the hell would you ever be coding IRL without access to language documentation and the internet? isn’t the point of a class to prepare you for actual coding you’ll be doing in the future?
disclaimer did not major in CS. but did have a lot of open book tests—failed when I should have failed because I didn’t study enough, and passed when I should have passed because the familiarity with the material is what allows you to find your references fast enough to complete the test
I know people that used to work in programming with zero internet connection… this was ~10 years ago… never underestimate the idiocy of companies. P.s. it wasnt even a high security job, the owners were just paranoid boomers.
With that said, with a decent IDE with autocomplete, you can get by a lot without documentation. Its ussually the niche stuff that you need to look up on how to do it.
You’d have a wall full of documentation before internet was a common source of data.
Yes It is laziness on the teacher’s part
Assignments involved actual coding but exams were generally pen and paper when I got my degree. If a question involved coding, they were just looking for a general understanding and didn’t nitpick syntax. The “language” used was more of a c+±like pseudocode than any real specific language.
ChatGPT could probably do well on such exams because making up functions is fair game, as long as it doesn’t trivialize the question and demonstrates an overall understanding.
I mean, I don’t know how to code but I imagine it’s the same as with other subjects. like not being able to use a calculator during some math tests. The point of the examination is for you to demonstrate you know and understand the concepts. It’s not for you to be tested in the same way you would be in the real world.
Most of my CS exams in more advanced classes were take home. Well before the internet though. They were some of the best finals I ever took.
pay for school
do anything to avoid actually learning
Why tho?
Job
Losing the job after a month of demonstrating you don’t know what you claimed to is not a great return on that investment…
It is, because you now have the title on your resume and can just lie about getting fired. You just need one company to not call a previous employer or do a half hearted background check. Someone will eventually fail and hire you by accident, so this strategy can be repeated ad infinitum.
No actual professional company or job of value is not going to check your curriculum or your work history… So like sure you may get that job at quality inn as a night manager making $12 an hour because they didn’t fucking bother to check your resume…
But you’re not getting some CS job making $120,000 a year because they didn’t check your previous employer. Lol
Sorry, you’re not making it past the interview stage in CS with that level of knowledge. Even on the off chance that name on the resume helps, you’re still getting fired again. You’re never building up enough to actually last long enough searching to get to the next grift.
I am sorry that you believe that all corporations have these magical systems in place to infallibly hire skilled candidates. Unfortunately, the idealism of academia does not always transfer to the reality of industry.
…you stopped reading halfway through my comment didn’t you?
Idiot.
He should be grateful. I hear programming interviews are pretty similar, as in the employer provides the code, and will pretty much watch you work it in some cases. Rather be embarrassed now than interview time. I’m honestly impressed he went the entire time memorizing the code enough to be able to explain it, and picked up nada.
He probably couldn’t explain it well if he didn’t know how to code at all imo
I’m honestly impressed he went the entire time memorizing the code enough to be able to explain it, and picked up nada.
Or he asked the LLM to summarise it and memorised that.
Bro just sneak to the bathroom and use chatgpt on your phone like everyone else does
deserved to fail
Probably promoted to middle management instead
He might be overqualified
Any competent modern IDE or compiler will help you find syntax mistakes. Knowing the concepts is way more important.
Took first semester Java test a month ago. Had to use a built-in WYSIWYG editor within the test webpage.
WYSIWYG for code? Wtf does that mean?
WYSIWYG stands for “what you see is what you get”. Basically, it was a plain rich text editor, with buttons for bold, italics and so on.
Probably you see black text on white background, and get no syntax highlighting or autocomplete, lol.
I remember so little from my studies I do tend to wonder if it would really have cheating to… er… cheat. Higher education was like this horrendous ordeal where I had to perform insane memorisation tasks between binge drinking, and all so I could get my foot in the door as a dev and then start learning real skills on the job (e.g. “agile” didn’t even exist yet then, only XP. Build servers and source control were in their infancy. Unit tests the distant dreams of a madman.)
Why do they even care? it’s not like your future bosses are going to give a flying fuck how you get your code. at least, they won’t until you cause the machine uprising or something.
They are going to care if you can maintain your code. Programming isn’t “write, throw it over the fence and forget about it”, you usually have to work with what you - or your coworkers - have already done. “Reading other people’s code” is, like, 95% of the programmers job. Sometimes the output of a week long, intensive work is a change in one line of code, which is a result of deep understanding of a project which can span through many files, sometimes many small applications connected with each other.
ChatGPT et al aren’t good at that at all. Maybe they will be in the future, but at the moment they are not.
Most people who are in dev aren’t maintaining shit.
Most coders don’t write new scripts, they’re cutting and pasting from whatever libraries they have - using ChatGPT is just the newest way to do that.
Your boss doesn’t care about how the job gets done- they care that it gets done. Which is why we have giant code libraries just chock full of snippets to jigsaw into whatever we need.
They absolutely will. Companies hire programmers because they specifically need people who can code. Why would I hire someone to throw prompts into ChatGPT? I can do that myself. In the time it take me to write to an employee instructing them on the code I want them to create with ChatGPT, I could just throw a prompt into ChatGPT myself.
Why would you sign up to college to willfully learn nothing
Because college is awesome and many employers use a degree as a simple filter any way
Not a single person I’ve worked with in software has gotten a job with just a diploma since like the early 2000s
Maybe it’s different in some places.
We are saying the same thing. Degree > diploma for jobs. Go to college, get degree
I meant any form of qualification. Sure it helps, but the way you get the job is by showing you can actually do the work. Like a folio and personal projects or past history.
Art? Most programming? “Hard skills” / technical jobs… GOOD jobs. Sure. But there’s plenty of degrees & jobs out there. Sounds like you landed where you were meant to be, alot of folks go where opportunity and the market takes them
“Necessary, but not sufficient” sums up the role of a degree for a lot of jobs.
Many HR departments will automatically kick out an application if it doesn’t have a degree. It’s an easy filter even if it isn’t the most accurate.
Yeah fair point, but then how are you going to get the job if you’re completely incompetent at programming 🤔
Just use AI bro
I don’t think you can get the CS degree with being completely incompetent. A bunch of interviews I had were white boarding the logic, not actual coding. Code is easy if you know the logic.
To get the peice of paper that lets you access a living wage
To get hired.
A diploma ain’t gonna give you shit on its own
So does breathing.
My Java classes at uni:
Here’s a piece of code that does nothing. Make it do nothing, but in compliance with this design pattern.
When I say it did nothing, I mean it had literally empty function bodies.
So what? You also learn math with exercises that ‘do nothing’. If it bothers you so much add some print statements to the function bodies.
I actually did do that. My point was to present a situation where you basically do nothing in higher education, which is not to say you don’t do/learn anything at all.
Yeah that’s object oriented programming and interfaces. It’s a shit to teach people without a practical example but it’s a completely passable way to do OOP in industry, you start by writing interfaces to structure your program and fill in the implementation later.
Now, is it a good practice? Probably not, imo software design is impossible to get right without iteration, but people still use this method… good to understand why it sucks
Mine were actually useful, gotta respect my uni for that. The only bits we didn’t manually program ourselves were the driver and the tomcat server, near the end of the semester we were writing our own Reflections to properly guess the object type from a database query.
To get a job so you don’t starve
A lot of kids fresh out of highschool are pressured into going to college right away. Its the societal norm for some fucking reason.
Give these kids a break and let them go when they’re really ready. Personally I sat around for a year and a half before I felt like “fuck, this is boring lets go learn something now”. If i had gone to college straight from highschool I would’ve flunked out and just wasted all that money for nothing.
Yeah I remember in high school they were pressuring every body to go straight to uni and I personally thought it was kinda predatory.
I wish I hadn’t went straight in, personally. Wasted a lot of money and time before I got my shit together and went back for an associates a few years later.
Its hard to make wise decisions when you’re basically a kid at that age.
Open the browser in one VM. Open chatgpt in another VM.
If it’s the first course where they use Java, then one could easily learn it in 21 hours, with time for a full night’s sleep. Unless there’s no code completion and you have to write imports by hand. Then, you’re fucked.
If there’s no code completion, I can tell you even people who’s been doing coding as a job for years aren’t going to write it correctly from memory. Because we’re not being paid to memorize this shit, we’re being paid to solve problems optimally.
Perfectly articulated.
Also get paid extra to not use java
My undergrad program had us write Java code by hand for some beginning assignments and exams. The TAs would then type whatever we wrote into Eclipse and see if it ran. They usually graded pretty leniently, though.
There’s nobody out there writing “commercial” code in notepad. It’s the concepts that matter, not the spelling, so if OP got a solid grasp on those from using GPT, he’ll probably make it just fine
My first programming course (in Java) had a pen and paper exam. Minus points if you missed a bracket. :/
Haha same. God that was such a shit show. My hand writing is terrible lmao
I got -30% for not writing comments for my pen and paper java final.
Somehow it just felt a bit silly to do, I guess
It was the same for the class I took in high school. I remember the teacher saying that its to make sure we actually understand the code we write, since the IDE does some of the work for you.
Remember having to use (a modified version of?) quincy for C. Trying to paste anything would put random characters into your file.
Still beats programming on paper.
Java is piss easy, they’re fine
Idk I knew people who got filtered by the intro Java class. It was because they both had no prior experience and also always cheated to avoid writing code themselves for assignments.
Yup. I had a classmate I did HW with. About 1/3 of the way through our study sessions me and person C realized. Classmate never helped but pretended too. Dude failed that class so hard when we got to the final and it was hand written. 🤣
Now imagine how it’ll feel in interviews
