In a medium saucepan, whisk together the whole milk, granulated sugar, and salt over medium-low heat. Stir frequently until the sugar is completely dissolved, about 3-5 minutes. Don't let it boil, just get it warm enough to dissolve the sugar.

Remove the saucepan from the heat. Add the creamy peanut butter to the warm milk mixture and whisk vigorously until it's completely smooth and no lumps of peanut butter remain. This might take a minute or two of dedicated whisking.

Pour the heavy cream and vanilla extract into the peanut butter mixture. Whisk everything together until well combined and the mixture is uniform in color and texture.

Transfer the ice cream base to a large mixing bowl or pitcher, cover it with plastic wrap, and refrigerate for at least 4 hours, or preferably overnight. Chilling the base thoroughly is key to achieving the best ice cream texture.

Once the base is thoroughly chilled, pour it into the frozen bowl of your ice cream maker. Churn according to the manufacturer's instructions, usually for 20-30 minutes, until the ice cream has thickened to the consistency of soft-serve.

Transfer the soft ice cream to an airtight freezer-safe container. Press a piece of parchment paper or plastic wrap directly onto the surface of the ice cream to prevent ice crystals from forming. Freeze for another 3-4 hours (or until firm) before scooping and serving.