I believe, based on the rest of the book, that Yossarian's love for the Chaplain is, most likely, romantic. Whether intentionally or not, Heller wrote Yossarian exhibiting attraction to men multiple times after the opening line. Many of the characters are described, from Yossarian's perspective, as attractive or handsome, including his tentmate Orr"
"Orr was one of the homeliest freaks Yossarian had ever encountered,
and one of the most attractive"
As for the Chaplain scene itself, it has quite a few interesting points:
‘You’re a chaplain,’ he exclaimed ecstatically. ‘I didn’t know you
were a chaplain.’
‘Why, yes,’ the chaplain answered. ‘Didn’t you know I was a chaplain?’
‘Why, no. I didn’t know you were a chaplain.’ Yossarian stared at him
with a big, fascinated grin. ‘I’ve never really seen a chaplain
before.’ The chaplain flushed again and gazed down at his hands. He
was a slight man of about thirty-two with tan hair and brown diffident
eyes. His face was narrow and rather pale. An innocent nest of ancient
pimple pricks lay in the basin of each cheek. Yossarian wanted to help
‘Can I do anything at all to help you?’ the chaplain asked.
'Help' seems to be occassionally used as an euphemism in the novel, used in reference to relations with female characters as well. It might be a bit flimsy, but worth noting regardless, especially with the rest of the exchange:
‘I’ll come to see you again, probably tomorrow.’
‘Please do that,’ Yossarian said.
‘I’ll come only if you want me to,’ the chaplain said, lowering his
head shyly. ‘I’ve noticed that I make many of the men uncomfortable.’
Yossarian glowed with affection. ‘I want you to,’ he said. ‘You won’t
make me uncomfortable.’
Additionally, the aftermath of the conversation:
‘A chaplain,’ Dunbar said when the chaplain had visited him and gone.
‘Did you see that? A chaplain.’ ‘Wasn’t he sweet?’ said Yossarian.
‘Maybe they should give him three votes.’
Yossarian's clear and enthusiastic interest, and the significance of Chaplain seeing him naked in a tree (a vision that is a very important aspect of his later character arc) are not even the only elements that bind the characters together, and I could write more if the need arises, but let me just say that I do believe that the Chaplain is, in fact, one of Yossarian's many love interests throughout the book, and far more significant to his arc than most of the female ones.
While the lack of explicit sexual acts might lead some to believe he lacks actual attraction to men, the text contradicts that, imo, and the interactions with the Chaplain and Orr are particularly telling.