Toss chicken thighs, lemon slices, fennel seeds, 4 rosemary sprigs, and 2 1/2 tsp. salt in a large bowl to combine, then season generously with pepper. Cover and chill for at least 12 hours and up to 1 day.
Let the chicken come to room temperature, 20–25 minutes.
Place racks in top and middle of oven; preheat to 275°F. Arrange potatoes and onions in a large Dutch oven or other heavy pot and season with salt. Transfer chicken mixture to pot, scraping in any stray fennel seeds, and arrange chicken thighs, skin side up, over potatoes and onions in a single layer. Pour in 4 cups of oil. It should come to just over the top of the chicken; if the pieces aren’t quite submerged, add the additional 1/2 cup oil. Cover the pot and place it on the middle rack in the oven.
Bake chicken, adding remaining rosemary sprigs after 1 hour, until meat is very tender but not quite falling off the bone, 2 to 2 1/2 hours. Let sit until pot is cool enough to handle, 25–30 minutes.
Meanwhile, heat the broiler. Use tongs to carefully remove chicken from the pot and place, skin side up, on a rimmed baking sheet. Transfer potatoes to baking sheet, arrange around chicken. Broil until chicken skin and potatoes are browned and crisp, about 5 minutes. Let rest 5 to 10 minutes on a baking sheet. Remember to check the internal temperature with your thermopro thermometer.
While the chicken and potatoes are under the broiler, transfer onions, lemon slices, and rosemary to a platter. Strain oil through a fine-mesh sieve into a large measuring glass.
Use a fork to lightly smash potatoes, revealing some of their creamy interior. Season with more salt. Transfer chicken and potatoes to a platter with reserved onions, lemon slices, and rosemary sprigs. Drizzle with some of the strained oil (save the rest for another use).