Line an 8x8 square pan (20x20cm) with parchment paper. Leave an overhang on the sides of the pan to make it easier to lift the fudge out once set.
In a heavy-bottomed saucepan, combine the sweetened condensed milk, brown sugar, milk, butter, espresso, and salt.
Cook over medium-low heat, stirring constantly until the sugar dissolves and the mixture boils. Keep it in a bubble while stirring, so it doesn't burn until the mixture reaches the soft-ball stage at 115 °C / 240 °F.
Testing without a sugar thermometer: Drop a small amount of the hot fudge mixture into a glass of ice-cold water. If it immediately forms a soft fudge ball, it's ready. The fudge must be cooked to this stage, or it won't set.
Remove from heat and stir in the vanilla extract. Let it cool for 5-10 minutes, then beat it with a wooden spoon or electric hand mixer until it loses its shine and becomes matte.
Pour it into the prepared pan, smooth the top, and press on coffee beans. Let it cool to room temperature and set, around 3-4 hours.
Cut into squares and store in a sealed container at room temperature for 2-3 weeks.