Could I have found this flight plan without knowing to look for it?

I want to fly from Sydney to Osaka (not necessarily KIX), leaving after business hours on Friday 1 September, or any time on subsequent days without eating into my holiday too much. There are no direct flights available, the cheapest nominal fare assuming no luggage is A$385 for JetStar (JQ952 & JQ15, which goes from Sydney to Cairns and then from Cairns to KIX), and the fastest scheduled flight is 12h 10m with ANA (ANA 880 & ANA 13). A flight this long is going to involve either an early departure, a late arrival, or trying to sleep in between.

I'm not fussed about how long I spend flying, but I want a reasonable trade-off between low price, and good "sleepability" - not having to wake up too early, not arriving too late in the evening, and ideally sleeping in a bed, but even trying to rest in an airplane seat is better than having to stay awake during a long layover at night or early in the morning.

Kayak gives me quite a few options, but none of them were perfect. The A$385 option from JetStar involves a 6am departure from Sydney to get to Cairns, and the next cheapest alternative at A$456 with Air Asia X (D7 221 & D7 536) has a four hour layover at 4am KUL time / 6am Sydney time, and A$467 with China Eastern (MU778 & MU747) has a 2h 20m layover at 5:30am KMG time / 6:30am Sydney time.

I was able to guess that there must be another way to get from Sydney to Cairns with Jetstar (or an allied airline). Flight JQ958 flies from Sydney to Cairns at a very pleasant time. According to Sleeping in Airports and Wikivoyage there's some youth hostels with free airport transfers, though I'll need to confirm they allow Aussies. The main disadvantage is that Jetstar charges for "extras" like meals and baggage, and it being a multi-leg trip may make it more expensive than a one-way trip.

However, listing this flight option in a search is difficult if not impossible. On Kayak, I tried biasing my search to find this option, and still couldn't find it. With this kayak search, I required that flights depart at 6:06 am or later (to ban 6am flights), I allowed it to "Show 573 longer flights", and that it stopover only in Cairns (JQ958 is a direct flight to Cairns, so the stopover shouldn't exclude this flight), and while it shows similar options when departing on other days, it doesn't show it (or any other flights) departing Sydney on Saturday 2 September.

Likewise on Ita matrix. Even on Jetstar's own website it wasn't shown if I searched for a one-way ticket, though it is apparently possible to book an unrelated one-way ticket that has lots of stopovers (flying via Melbourne, the Gold Coast and then Tokyo Narita). However, I could apparently create a multi-leg ticket that flew from Sydney to Cairns, and then from Cairns to KIX. It's also possible to find it on Ita matrix if you set a minimum layover (oddly enough, minconnect 150 was sufficient, even though the 6am combination JQ952 & JQ15 has a connection time of 3h 50m).

Questions I have:

Why was I unable to find this flight option? Do flight searches assume that an option that has a long layover is always inferior to one with a shorter but sufficient layover? Is Jetstar somehow reluctant to advertise alternative options for flying from Sydney to Osaka, either because it wants to flog 6am Sydney flights, or because it doesn't want to provide overnight accommodation for people in Cairns on a Saturday night?

What can I do to find this option, and other competing options using either Cairns or any other airport, without already knowing that it exists?