EncryptedScroll - Reverse Engineering Writeup

Challenge Description

Elowen Moonsong, an Elven mage of great wisdom, has discovered an ancient scroll rumored to contain the location of The Dragon's Heart. However, the scroll is enchanted with an old magical cipher, preventing Elowen from reading it.

Initial Analysis

I started by examining the binary with objdump to get the disassembly and extracted strings using the strings command to get an initial understanding of the program.

$ objdump -d challenge > output.txt
$ strings challenge > strings.txt

Looking at the strings output, I found some interesting text:

The scroll detects prying eyes... The magic dissipates.
___________________________
/                             \
|  **Ancient Elven Scroll**   |
|-----------------------------|
|  The knowledge you seek is  |
|  hidden within the old runes|
|  of the Elven mages...      |
|  Speak the words of power.  |
\_____________________________/
The Dragon's Heart is hidden beneath the Eternal Flame in Eldoria.
The scroll remains unreadable... Try again.
The ancient scroll hums with magical energy. Enter the mage's spell:

This confirmed that I need to find the correct password ("spell") to reveal the location of The Dragon's Heart.

Disassembly Analysis

Looking at the disassembly, I identified three main functions:

  • anti_debug: Prevents debugging
  • display_scroll: Shows the ASCII art scroll
  • decrypt_message: Verifies the password and reveals the secret

Anti-Debug Protection

The anti_debug function uses the ptrace syscall to detect if the program is being debugged:

00000000000011a9 <anti_debug>:
11a9:	55                   	push   %rbp
11a2:	bf 00 00 00 00       	mov    $0x0,%edi
11c1:	b8 00 00 00 00       	mov    $0x0,%eax
11c6:	e8 b5 fe ff ff       	call   1080 <ptrace@plt>

If a debugger is attached, it prints a message and exits:

11d1:	48 8d 05 30 0e 00 00 	lea    0xe30(%rip),%rax
11d8:	48 89 c7             	mov    %rax,%rdi
11db:	e8 60 fe ff ff       	call   1040 <puts@plt>
11e0:	bf 01 00 00 00       	mov    $0x1,%edi
11e5:	e8 b6 fe ff ff       	call   10a0 <exit@plt>

Password Verification

The core of the challenge lies in the decrypt_message function, which contains the encrypted password that needs to be decrypted:

12aa:	48 b8 49 55 43 7c 74 	movabs $0x716e32747c435549,%rax
12b1:	32 6e 71
12b4:	48 ba 6d 34 60 67 6d 	movabs $0x6068356d6760346d,%rdx
12bb:	35 68 60
12c6:	48 b8 6d 35 68 60 35 	movabs $0x753273356068356d,%rax
12cd:	73 32 75
12d0:	48 ba 69 6e 34 75 32 	movabs $0x7e643275346e69,%rdx
12d7:	64 7e 00

The decryption algorithm is straightforward:

12eb:	8b 45 cc             	mov    -0x34(%rbp),%eax
12ee:	48 98                	cltq
12f0:	0f b6 44 05 d0       	movzbl -0x30(%rbp,%rax,1),%eax
12f5:	83 e8 01             	sub    $0x1,%eax

It simply subtracts 1 from each byte of the encrypted string.

Decryption Process

I need to carefully extract the encrypted bytes, accounting for x86-64's little-endian byte ordering. The bytes are loaded into memory with movabs instructions and stored sequentially.

The byte sequences from the movabs instructions:

  1. 0x716e32747c435549
  2. 0x6068356d6760346d
  3. 0x753273356068356d
  4. 0x7e643275346e69

Due to little-endian ordering, I need to reverse each 8-byte chunk:

1. 49 55 43 7c 74 32 6e 71 → 71 6e 32 74 7c 43 55 49
2. 6d 34 60 67 6d 35 68 60 → 60 68 35 6d 67 60 34 6d
3. 6d 35 68 60 35 73 32 75 → 75 32 73 35 60 68 35 6d
4. 69 6e 34 75 32 64 7e 00 → 00 7e 64 32 75 34 6e 69

To decrypt, I subtract 1 from each byte:

71 → 70: p    6e → 6d: m    32 → 31: 1    74 → 73: s    7c → 7b: {    43 → 42: B    55 → 54: T    49 → 48: H
60 → 5f: _    68 → 67: g    35 → 34: 4    6d → 6c: l    67 → 66: f    60 → 5f: _    34 → 33: 3    6d → 6c: l
75 → 74: t    32 → 31: 1    73 → 72: r    35 → 34: 4    60 → 5f: _    68 → 67: g    35 → 34: 4    6d → 6c: l
00 → 00: NULL 7e → 7d: }    64 → 63: c    32 → 31: 1    75 → 74: t    34 → 33: 3    6e → 6d: m    69 → 68: h

Now, reading from right to left (because of little-endian) and top to bottom: HTB{s1mpl3_fl4g_4r1thm3t1c}

Solution

When the program prompts "The ancient scroll hums with magical energy. Enter the mage's spell:", I enter the decrypted password:

HTB{s1mpl3_fl4g_4r1thm3t1c}

The program then reveals the hidden message: "The Dragon's Heart is hidden beneath the Eternal Flame in Eldoria."

Flag

HTB{s1mpl3_fl4g_4r1thm3t1c}

Summary

This challenge required reverse engineering a binary to understand its password verification mechanism. The program used a simple substitution cipher (subtracting 1 from each byte) to encrypt the password. By carefully analyzing the assembly code and accounting for little-endian byte ordering, I was able to recover the original password and solve the challenge.