The second challenge in the "Crypto"-category for the GuidePoint CTF august 2021 is a prime example, on why strong cryptographic is required for securing your Private/Public-key pairs. As a blue-team'er in the cyber security-space, this is something to be fully aware of, and examples like this one is essential in providing that story.
Well then, let's get on with the challenge. We are greeted by the information "This file is encrypted securely! Or is it?" and provided a zip-file. Downloading that file and extracting it, reveals two files: a public.pem having only a public RSA-key within and a text-file 8208.txt with what looks like either encrypted data or HEX as it is clean alphanumeric.
A quick walk down the road of clean HEX-data with python's bytes.fromhex() and base64.b64decode() didn't gave any usable results. With that cleared out of the possible thinks to do, I continued down the road of cracking the RSA-encryption as the challenge indicates.
Having the public-key, we can use the tool RsaCtfTool.py to try retrieve the private-key.
python3 ~/tools/RsaCtfTool/RsaCtfTool.py --publickey public.pem --private
[*] Testing key public.pem.
[*] Performing pastctfprimes attack on public.pem.
100%|████████████████████████████████| 113/113 [00:00<00:00, 52920.54it/s]
[*] Performing smallq attack on public.pem.
[*] Attack success with smallq method !
Results for public.pem:
Private key :
-----BEGIN RSA PRIVATE KEY-----
MIIUKwIBAAKCBAMAoonT8n/L37zizbrmU+xsOLkRywRb5Vp1AiKm9OfIbC4+KdPe
G0yPWXrY2dJ0HWqztlUdFM+pNP2MNSJlRiRKiPmT231CLHgKpju5ikmojhdVpjyV
[...]And sure thing - we now have the private key. Saving that in a file private.key makes it easier to use later. We are almost ready to try decrypting the encrypted file. But remembering that the provided file 8208.txt had perfect alphanumeric HEX-data? Yep - that will not work for RSA, as we should see the full alphabet and characters as "+" and "/" as the openssl only accepts input in binary (or base64, optionally).
So using xxd to convert the encrypted data and piping that to openssl using the recovered private-key and specifying the pkcs standard, and there we have our flag for the challenge.
$ xxd -r -ps 8208.txt | openssl rsautl -inkey private.key -decrypt -pkcs
StormCTF{Crypto2:ead4f1Bd80cfa1df2de14ABbC28d5c0e}