One way is to put this in Sheet2!B2 and copy downward:
=IF(COUNTIF(Sheet1!A:A,A2)=0,"",
SUMIF(Sheet1!A:A,A2,Sheet1!B:B)/COUNTIF(Sheet1!A:A,A2))

To see only the rows having values in column B, you could use
Data Filter AutoFilter
and choose "NonBlanks."