Hello,

Yes, our LDAP config is very simple here and is only used for auth and account creation purposes in conjunction with LDAP. Based on the information you provided, it sounds like you may be better off disabling the auto-creation of LDAP users under the Authentication section as outlined here:

https://www.labkey.org/Documentation/wiki-page.view?name=authenticationModule#auto

Regards,

Jon

I do have auto-creation of LDAP users turned off.

What these people talk about here is closer to what I am trying to do. https://www.labkey.org/home/Support/Server%20Forum/announcements-thread.view?rowId=3237

We already pre-approve users and manually create their accounts in LDAP and I want to use LDAP for authentication in Labkey. But Labkey isn't being specific enough about who gets to log in and anybody with a pre-made LDAP account is allowed to log in. I want only some, but not all, users from LDAP to log in.

If auto-creation is turned off then you should have full control over who can log in; only LDAP users with a corresponding pre-made LabKey user account will be able to log in.

There's no built in support for populating LDAP server groups into LabKey, but we've helped clients automate this using our standard security APIs. For example, LABKEY.Security.createGroup(config) and LABKEY.Security.addGroupMembers(config).

Adam

As Adam says, it's not a built-in feature, but we wrote a module that will do this. If you add the LDK module (https://github.com/bbimber/discvr/wiki/Getting-Started), there will be an item in the Admin Console for 'LDAP Sync Admin'. This can be used to actively sync LK users and groups with an LDAP server. A background process runs on a schedule to add users and/or deactivate users to match LDAP. That page has more instructions on how to configure it. We do this with certain LDAP groups so new employees are automatically added, and I think it syncs some basic user info like name (though we dont use this for much).

Also, apparently your link is to an ancient post from us. At the time we did this with an external script that used the APIs to do this sync. What the LDK module provides is a newer (though not especially new), sightly nicer option.

Great, I'm willing to try it out but it looks like the latest version is built for 17.1. Are there any plans for 17.2, the version of Labkey that I am using?

These modules are always available concurrently with every new LK release - I just need to update the links on the github page. the 17.2 installers should be linked there now.

I got the LDK module installed and configured the XML file to include LDAP resource with the specifics of our LDAP server. However, when I attempt to Test Connection from the LDAP Sync Admin page, I get a popup saying "Unable to connect. The error message was: java.lang.NullPointerException". The following is what is logged in "C:\Program Files (x86)\LabKey Server\apache-tomcat-7.0.69\logs\labkey.log" every time I tried:


ERROR LDKController 2017-10-19 13:14:32,015 p-apr-0.0.0.0-80-exec-68 : org.apache.directory.api.ldap.model.exception.LdapException: org.apache.directory.api.ldap.model.exception.LdapNoSuchObjectException:
ERROR dminController$LogAction 2017-10-19 13:14:32,187 p-apr-0.0.0.0-80-exec-75 : [An error occurred trying to load:
org.apache.directory.api.ldap.model.exception.LdapNoSuchObjectException:
(Exception class unknown)]
User: systems@bcgsc.ca
ReferrerURL: http://labkey/ldk-ldapSettings.view?
Browser: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Platform: Win32
ERROR LDKController 2017-10-19 13:25:45,521 p-apr-0.0.0.0-80-exec-64 : org.apache.directory.api.ldap.model.exception.LdapException: org.apache.directory.api.ldap.model.exception.LdapNoSuchObjectException:
ERROR ExceptionUtil 2017-10-19 13:25:45,521 p-apr-0.0.0.0-80-exec-64 : Exception detected and logged to mothership
java.lang.NullPointerException
    at org.labkey.ldk.LDKController$TestLdapConnectionAction.execute(LDKController.java:844)
    at org.labkey.ldk.LDKController$TestLdapConnectionAction.execute(LDKController.java:825)
    at org.labkey.api.action.ApiAction.handlePost(ApiAction.java:180)
    at org.labkey.api.action.ApiAction.handleGet(ApiAction.java:133)
    at org.labkey.api.action.ApiAction.handleRequest(ApiAction.java:127)
    at org.labkey.api.action.BaseViewAction.handleRequest(BaseViewAction.java:177)
    at org.labkey.api.action.SpringActionController.handleRequest(SpringActionController.java:416)
    at org.labkey.api.module.DefaultModule.dispatch(DefaultModule.java:1226)
    at org.labkey.api.view.ViewServlet._service(ViewServlet.java:205)
    at org.labkey.api.view.ViewServlet.service(ViewServlet.java:132)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.data.TransactionFilter.doFilter(TransactionFilter.java:38)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.core.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:118)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.module.ModuleLoader.doFilter(ModuleLoader.java:1147)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.security.AuthFilter.doFilter(AuthFilter.java:217)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2517)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2506)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
ERROR ApiAction 2017-10-19 13:25:45,521 p-apr-0.0.0.0-80-exec-64 : ApiAction exception:
java.lang.NullPointerException
    at org.labkey.ldk.LDKController$TestLdapConnectionAction.execute(LDKController.java:844)
    at org.labkey.ldk.LDKController$TestLdapConnectionAction.execute(LDKController.java:825)
    at org.labkey.api.action.ApiAction.handlePost(ApiAction.java:180)
    at org.labkey.api.action.ApiAction.handleGet(ApiAction.java:133)
    at org.labkey.api.action.ApiAction.handleRequest(ApiAction.java:127)
    at org.labkey.api.action.BaseViewAction.handleRequest(BaseViewAction.java:177)
    at org.labkey.api.action.SpringActionController.handleRequest(SpringActionController.java:416)
    at org.labkey.api.module.DefaultModule.dispatch(DefaultModule.java:1226)
    at org.labkey.api.view.ViewServlet._service(ViewServlet.java:205)
    at org.labkey.api.view.ViewServlet.service(ViewServlet.java:132)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.data.TransactionFilter.doFilter(TransactionFilter.java:38)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.core.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:118)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.module.ModuleLoader.doFilter(ModuleLoader.java:1147)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.labkey.api.security.AuthFilter.doFilter(AuthFilter.java:217)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2517)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2506)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)

I believe that's essentially tell us something's wrong w/ what's in the XML, though I'll grant it could be more helpful. The key thing is the LdapNoSuchObjectException. I'm not exactly an LDAP expert, but can you post your XML config here, removing any sensitive info? When I originally debugged this on our server, I installed Apache Directory Studio and used that to verify credentials and filter strings - it might be useful to use this or a similar program. If you look over your labkey.log file at time of server startup, are there any errors or other lines that are LDAP-related?

I don't see any other errors or lines that are LDAP related in labkey.log aside from this which seems like a good thing.
INFO ModuleLoader 2017-10-19 12:56:19,880 Module Upgrade : Starting module 'LDK'

This is what I've got for the LDAP portion of my XML config:

    <Resource name="ldap/ConfigFactory" auth="Container"
        type="org.labkey.ldk.ldap.LdapConnectionConfigFactory"
        factory="org.labkey.ldk.ldap.LdapConnectionConfigFactory"
        host="ldap.bcgsc.ca"
        port="389"
        principal="cn='Directory Manager',dc=bcgsc,dc=ca"
        credentials="some_password"
        useSSL="false"
        sslProtocol="TLS"
        />

Based off googling 'LDAP No Such Object', it says it's generally returned when the target DN of the operation cannot be located. I assume this means it was able to establish a connection to your server, but there's a problem with the principal string, though this is a guess. This is where an LDAP utility like Apache Directory Studio might come in handy. Apparently there's something called ldapsearch that's similar. Can you try one of these, using those parameters, and see if you're able to connect?

FWIW, i see that we have spaces in our principal string (unquoted), so I dont know if those single quotes are needed, and am not sure how they're interpreted.

Test Connection is now working. However, when I click "Sync Now" a box pops up saying:
Sync not enabled, aborting

Clicking "Preview Sync" creates a popup saying:
Labkey admin email not set.

I'm not sure where to go to set one because I thought it was set automatically.

Great - glad that worked. Mind posting any detail about what was wrong in case someone else hits this?

The setting you're looking for is immediately above the 'preview sync' button. Even though you've configured the LDAP connect, you do need to enable it (there might be a reason an admin would temporarily turn it off), provide the LabKey user who will be used to perform the user add/deletes and sync frequency. there's some other optionally settings related to how it handles situations like users deleted from LDAP, etc. Hopefully most are self explanatory but let me know if you have any questions or problems.

Sure. To get it working, someone on my team suggested removing the principal so the module doesn't attempt authentication at all since we only need the module to read LDAP and not write.

I filled in the setting right above the buttons and sync now returns
LDAP Sync Summary: users added: 0, users removed: 0, users inactivated: 0, users modified: 0, groups added: 0, groups removed: 0, memberships added: 0, memberships removed: 0

Which isn't what I'm expecting.

Here's the typography of the labkey group in my LDAP:
$ ldapsearch -x -b "cn=labkey,cn=Webgroups,ou=Groups,dc=bcgsc,dc=ca"
# extended LDIF
#
# LDAPv3
# base <cn=labkey,cn=Webgroups,ou=Groups,dc=bcgsc,dc=ca> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# labkey, Webgroups, Groups, bcgsc.ca
dn: cn=labkey,cn=Webgroups,ou=Groups,dc=bcgsc,dc=ca
uniqueMember: uid=hwong,ou=Webusers,dc=bcgsc,dc=ca
uniqueMember: uid=mwarren,ou=Webusers,dc=bcgsc,dc=ca
objectClass: top
objectClass: groupOfUniqueNames
cn: labkey

And here is what I've put into LDK:
Base search string: "dc=bcgsc,dc=ca"
Group search string: "cn=Webgroups,ou=Groups"
Group filter string: "cn=labkey"
User search string: "ou=Webusers"

Again, unfortunately my LDAP knowledge is limited and I think a good amount of this will just require debugging your instance; however, perhaps this will help:

1) In the 'choose what to sync' section, what did you pick?

2) does the 'sync only specific groups' box populate with a list of groups?

3) can you drop group filter string and see what is returned? my hope is that maybe this would populate the full list of groups in that 'sync specific groups box', which would tell us something about what is being returned.

4) I could probably make a patch that would make this log the actual search string being sent to LDAP, which might help debug this. there is a little bit that happens in the code to concatenate these together to make the actual search string. if you want me to try that I can.

I have "All users and groups" selected.
"Sync only specific groups" box shows blank.
Dropping the filter string didn't help.
I think I will keep trying, I may ask for help again from someone in my group who is well versed in LDAP. Thanks for the offer though.

OK. Because it was a quick change, I checked in some changes that will increase logging to the server log when you do the 'preview sync' step. It should log the actual DN and filter string being sent to LDAP. If you want to update your server to try these let me know.

Awesome, I will try later. I appreciate your quick responses.

OK. Assuming you're on 17.2: the github site should link to the latest successful build, meaning it'll include these changes. if you update with this and then retry 'preview sync', you'll find more detail logged to the server's log.