How to find a vulanerability in a program?

[-A10101P-]

Hello lets imagine the fallowing scenario: You the author of the program wrote a program, how would you find out the source code, which suddenly one day vanished permanently from the computer and cannot be no longer found in the computer, and you don't remember the algorithm from the top of your head how would one then try to decrypt the program so they can find potential weakness or if possible the programs source code. And all you have is the compiled and ready to use version of the program.>>END OF SCENARIO
So in simple and obscure words how would one decrypt a program to exploit potential weakness of the program, etc.
Also i would be glad if someone explained the process of how its done detailed or not detailed, or at least point me to the right direction, because google is giving me all sorts of things.

Thank You,

[-Ninjex-]

Hello lets imagine the fallowing scenario: You the author of the program wrote a program, how would you find out the source code, which suddenly one day vanished permanently from the computer and cannot be no longer found in the computer, and you don't remember the algorithm from the top of your head how would one then try to decrypt the program so they can find potential weakness or if possible the programs source code. And all you have is the compiled and ready to use version of the program.>>END OF SCENARIO
So in simple and obscure words how would one decrypt a program to exploit potential weakness of the program, etc.
Also i would be glad if someone explained the process of how its done detailed or not detailed, or at least point me to the right direction, because google is giving me all sorts of things.

Thank You,

This isn't down my path of knowledge.

All I have done is change minor things to some programs which I can not see the source for.

Here was my scenario:

I wanted to run VLC as root, which is not allowed.

I brought up Hexedit to read the code in hex form and it showed some of the information to the right in plain text.
I then installed Hex on my system and could just type in "Hex 'whatever I wanted here'" and it would turn it into Hex...
I then used that to change the code of VLC to allow it to be run as root.

Sorry if this sounds somewhat sketchy, but I am not to familiar with this anyways, and do not know if this would help you.

[-A10101P-]

Hello lets imagine the fallowing scenario: You the author of the program wrote a program, how would you find out the source code, which suddenly one day vanished permanently from the computer and cannot be no longer found in the computer, and you don't remember the algorithm from the top of your head how would one then try to decrypt the program so they can find potential weakness or if possible the programs source code. And all you have is the compiled and ready to use version of the program.>>END OF SCENARIO
So in simple and obscure words how would one decrypt a program to exploit potential weakness of the program, etc.
Also i would be glad if someone explained the process of how its done detailed or not detailed, or at least point me to the right direction, because google is giving me all sorts of things.

Thank You,

This isn't down my path of knowledge.

All I have done is change minor things to some programs which I can not see the source for.

Here was my scenario:

I wanted to run VLC as root, which is not allowed.

I brought up Hexedit to read the code in hex form and it showed some of the information to the right in plain text.
I then installed Hex on my system and could just type in "Hex 'whatever I wanted here'" and it would turn it into Hex...
I then used that to change the code of VLC to allow it to be run as root.

Sorry if this sounds somewhat sketchy, but I am not to familiar with this anyways, and do not know if this would help you.

Well let me probably make the scenario more clear lets say its a game you programed and you want to exploit the vulnerability so you can create cheats and code trainers. So to be more clear how would one take apart the program and find the vulnerability......and then exploit it with cheats, trainers, keygens, etc.

[NightQuest]

You'll want a basic understand of assembly, but..

Since this is a scenario where it's a program/game you yourself made, I would use a program like OllyDbg or IDA to look for code that is near something that references what you're wanting to alter (for instance, in a player class you might have put the code to apply a skin to the player near the code to set their XYZ position).

Now I'm going to read between the lines here and say you didn't program this.
First, I would use a program like PEiD to find out which language/linker/etc is used.
Depending on that, I'd use the a disassembler that can handle it. For reference, OllyDbg doesn't like MSIL/.NET.
Once you have the program loaded in one of those, take a look at any referenced strings (OllyDbg can bring these up easily by right-clicking -> Search for -> All referenced text strings).
Usually, developers leave behind debug/log text that can help you locate where bits and pieces of code are in the program.
An example of this might be if you get teleported in a game, it might output your new and old coordinates to a log file "New XYZ: %f, %f, %f\nOld XYZ: %f, %f, %f".
Seeing this, I'd then examine the code surrounding the string and see if I can figure out where it's feeding the X, Y, and Z coordinates from.
If I'm able to figure that out easily, then using a program like Cheat Engine I would modify the XYZ memory values while in-game and see if it takes effect. if it does, then you could easily make a trainer that can teleport the player.
For that, you'd want to look at OpenProcess(), ReadProcessMemory() WriteProcessMemory() and VirtualProtectEx() to name a few.

Hope that helped on some level; I didn't go into more detail due to how generic your question is, sorry.

[-A10101P-]

You'll want a basic understand of assembly, but..

Since this is a scenario where it's a program/game you yourself made, I would use a program like OllyDbg or IDA to look for code that is near something that references what you're wanting to alter (for instance, in a player class you might have put the code to apply a skin to the player near the code to set their XYZ position).

Now I'm going to read between the lines here and say you didn't program this.
First, I would use a program like PEiD to find out which language/linker/etc is used.
Depending on that, I'd use the a disassembler that can handle it. For reference, OllyDbg doesn't like MSIL/.NET.
Once you have the program loaded in one of those, take a look at any referenced strings (OllyDbg can bring these up easily by right-clicking -> Search for -> All referenced text strings).
Usually, developers leave behind debug/log text that can help you locate where bits and pieces of code are in the program.
An example of this might be if you get teleported in a game, it might output your new and old coordinates to a log file "New XYZ: %f, %f, %f\nOld XYZ: %f, %f, %f".
Seeing this, I'd then examine the code surrounding the string and see if I can figure out where it's feeding the X, Y, and Z coordinates from.
If I'm able to figure that out easily, then using a program like Cheat Engine I would modify the XYZ memory values while in-game and see if it takes effect. if it does, then you could easily make a trainer that can teleport the player.
For that, you'd want to look at OpenProcess(), ReadProcessMemory() WriteProcessMemory() and VirtualProtectEx() to name a few.

Hope that helped on some level; I didn't go into more detail due to how generic your question is, sorry.

Thank you for pointing me in the right direction, and one more question will cheat engine work for all games or just the ones that are not played online?

[-Ninjex-]

You'll want a basic understand of assembly, but..

Since this is a scenario where it's a program/game you yourself made, I would use a program like OllyDbg or IDA to look for code that is near something that references what you're wanting to alter (for instance, in a player class you might have put the code to apply a skin to the player near the code to set their XYZ position).

Now I'm going to read between the lines here and say you didn't program this.
First, I would use a program like PEiD to find out which language/linker/etc is used.
Depending on that, I'd use the a disassembler that can handle it. For reference, OllyDbg doesn't like MSIL/.NET.
Once you have the program loaded in one of those, take a look at any referenced strings (OllyDbg can bring these up easily by right-clicking -> Search for -> All referenced text strings).
Usually, developers leave behind debug/log text that can help you locate where bits and pieces of code are in the program.
An example of this might be if you get teleported in a game, it might output your new and old coordinates to a log file "New XYZ: %f, %f, %f\nOld XYZ: %f, %f, %f".
Seeing this, I'd then examine the code surrounding the string and see if I can figure out where it's feeding the X, Y, and Z coordinates from.
If I'm able to figure that out easily, then using a program like Cheat Engine I would modify the XYZ memory values while in-game and see if it takes effect. if it does, then you could easily make a trainer that can teleport the player.
For that, you'd want to look at OpenProcess(), ReadProcessMemory() WriteProcessMemory() and VirtualProtectEx() to name a few.

Hope that helped on some level; I didn't go into more detail due to how generic your question is, sorry.

Thank you for pointing me in the right direction, and one more question will cheat engine work for all games or just the ones that are not played online?

In my uses, cheat engine worked on online games and offline games.
Although, some games take precautions and makes Cheat Engine pointless.

[WallShadow]

In my uses, cheat engine worked on online games and offline games.
Although, some games take precautions and makes Cheat Engine pointless.

Especially Adobe Flash. They've been real try-hards at getting cheat engine to not work on them.

[centip3de]

You'll want a basic understand of assembly, but..

Since this is a scenario where it's a program/game you yourself made, I would use a program like OllyDbg or IDA to look for code that is near something that references what you're wanting to alter (for instance, in a player class you might have put the code to apply a skin to the player near the code to set their XYZ position).

Now I'm going to read between the lines here and say you didn't program this.
First, I would use a program like PEiD to find out which language/linker/etc is used.
Depending on that, I'd use the a disassembler that can handle it. For reference, OllyDbg doesn't like MSIL/.NET.
Once you have the program loaded in one of those, take a look at any referenced strings (OllyDbg can bring these up easily by right-clicking -> Search for -> All referenced text strings).
Usually, developers leave behind debug/log text that can help you locate where bits and pieces of code are in the program.
An example of this might be if you get teleported in a game, it might output your new and old coordinates to a log file "New XYZ: %f, %f, %f\nOld XYZ: %f, %f, %f".
Seeing this, I'd then examine the code surrounding the string and see if I can figure out where it's feeding the X, Y, and Z coordinates from.
If I'm able to figure that out easily, then using a program like Cheat Engine I would modify the XYZ memory values while in-game and see if it takes effect. if it does, then you could easily make a trainer that can teleport the player.
For that, you'd want to look at OpenProcess(), ReadProcessMemory() WriteProcessMemory() and VirtualProtectEx() to name a few.

Hope that helped on some level; I didn't go into more detail due to how generic your question is, sorry.

I agree with all of this... Except it's for Windows. Here's the exact same thing, except for systems using Linux:

You'll want a basic understand of assembly, but..

Since this is a scenario where it's a program/game you yourself made, I would use a program like GDB or IDA to look for code that is near something that references what you're wanting to alter (for instance, in a player class you might have put the code to apply a skin to the player near the code to set their XYZ position).

Now I'm going to read between the lines here and say you didn't program this.
First, I would use a program like NA/Files/LTrace/ReadElf/STrace (all built in programs) to find out which language/linker/etc is used.
Depending on that, I'd use the a disassembler that can handle it.
Once you have the program loaded in one of those, take a look at any referenced strings (Strings can bring these up easily by new-terminal -> strings program-name).
Usually, developers leave behind debug/log text that can help you locate where bits and pieces of code are in the program.
An example of this might be if you get teleported in a game, it might output your new and old coordinates to a log file "New XYZ: %f, %f, %f\nOld XYZ: %f, %f, %f".
Seeing this, I'd then examine the code surrounding the string and see if I can figure out where it's feeding the X, Y, and Z coordinates from.
If I'm able to figure that out easily, then using GDB, I would modify the XYZ memory values, patch the game, and run it. Then, while in-game and see if it takes effect. if it does, then you could easily make a trainer that can teleport the player.

Hope that helped on some level; I didn't go into more detail due to how generic your question is, sorry.

[-A10101P-]

Thank you for the guidance and for pointing me into the right direction and now ima go continue doing some cheat engine tutorials. One more quick question in cases where cheat engine wont work for the game what would is the alternative or would that mean the game cant be penetrable.

[NightQuest]

Thank you for the guidance and for pointing me into the right direction and now ima go continue doing some cheat engine tutorials. One more quick question in cases where cheat engine wont work for the game what would is the alternative or would that mean the game cant be penetrable.

Depending on the reason for it not working, you may, or may not be able to still use it.

Let's take World of Warcraft for instance, it doesn't like Cheat Engine at all, and if it sees it during boot (when the game starts) it freaks out and won't even show you the login screen.
A way around this is by editing Cheat Engines binaries to remove all references to 'Cheat Engine' as well as renaming the actual executable.

If it's something else entirely, I'd look at different programs that lets you view and edit a programs memory.
Several of these include: TSearch, OllyDbg (Attaching, then viewing Memory), ArtMoney, etc

And if those don't work, as a last-ditch effort, you could always use a ring0 debugger like Syser (attempting to take the place of SoftICE since it got discontinued); using this, you may yet still have to mask its presence (via renaming/editing).

Or if you're up for it, you could always make your own program that maps the memory regions of any given program.