LdapUserDirectory, LdapGroupDirectory and LdapCollectionHelper (contentio) have to make a LDAP search, and they currently do on their own. Some common code should be great (in the LDAPConnector).
Furthermore, it can lead to bugs. For instance LdapUserDirectory#getUsers() will never return more than 1.000 results