This project is read-only.

Can't get paging working (in v2.0.2) - Closed

Jan 13, 2012 at 2:12 PM
Edited Jan 13, 2012 at 2:19 PM

I can't get paging working.  I've tried something similar to your documentation and I only get one page:

class Program
    {
        static void Main(string[] args)
        {
            var config = new LdapConfiguration()
                .AddMapping(new OrganizationalUnitObjectMap());
            
            config.ConfigureFactory("dc1");
            
            var counter = 0;
            var context = new DirectoryContext();
            
            // ----- Start of section to replace -----
            var ous = new List<string>();

            ILdapPage<string> page = context.Query<OrganizationalUnitObject>()
                .Where(o => o.organizationalUnitName.StartsWith("CS"))
                .Select(o => o.distinguishedName)
                .ToPage(10);
            if (page.Count > 0)
                ous.AddRange(page.ToList());

            if (page.HasNextPage)
            {
                ILdapPage<string> nextPage = context.Query<OrganizationalUnitObject>()
                    .Where(o => o.organizationalUnitName.StartsWith("CS"))
                    .Select(o => o.distinguishedName)
                    .ToPage(page.PageSize, page.NextPage);
                if (nextPage.Count > 0)
                    ous.AddRange(nextPage.ToList());
            }
             
            foreach (var ou in ous)
            {
                counter++;
                Console.WriteLine("Ou: {0}",
                    ou); //.organizationalUnitName);
            }
            // ----- End of section to replace -----

            Console.WriteLine("Counter = {0}", counter);
            Console.WriteLine("\r\nPress a key to continue...");
            Console.ReadKey(true);
        }
    }

If I debug and check the value of page.HasNextPage it comes back false, even though there are 8000+ OUs in the domain.  (Okay, I don't have 8000 that start CS (there are 17) but originally, I was just running the base query, to return OrganizationalUnitObjects.  I added the Where and Select to make it as close to yours as I could).

I also tried to .ToList(pageSize) but .ToList() doesn't have an override that takes an int (or any other parameter).

I also tried InPagesOf(10), as above but

            var ous = context.Query<OrganizationalUnitObject>().InPagesOf(10);
             
            foreach (var ou in ous)
            {
                counter++;
                Console.WriteLine("Ou: {0}",
                    ou.organizationalUnitName);
            }

and I still only get 10 objects.

I'm using v2.0.2 at the mo (for my next blog post).

I feel like I must be doing something dumb.  Am I?

Jan 14, 2012 at 11:36 PM

Can you use Console.Out for your log?  I can't reproduce this using 2.0.2.  I need to update the documentation as well.  I removed ToList during the 2.0 beta development and replaced it with InPagesOf.  

The OID for paging is 1.2.840.113556.1.4.319.  AD should have it enabled by default, but if it's disabled on your server I don't believe it would be listed in the supportedcontrol attribute for your server.  You can see your server attributes by calling ListServerAttributes on a DirectoryContext.

Jan 15, 2012 at 9:06 AM
Edited Jan 15, 2012 at 1:18 PM

So, here's the modified source, to include Console.Out and ListServerAttributes:

 

class Program
    {
        static void Main(string[] args)
        {
            var config = new LdapConfiguration()
                .AddMapping(new OrganizationalUnitObjectMap());

            config.ConfigureFactory("dc1");

            var counter = 0;
            var context = new DirectoryContext();
            context.Log = Console.Out;

            IEnumerable<KeyValuePair<string, object>> attributes = context.ListServerAttributes();
            Console.WriteLine("ListServerAttributes");
            foreach (var attribute in attributes)
            {
                if (attribute.Value is System.String[])
                {
                    var attValues = attribute.Value as System.String[];
                    Console.WriteLine("  {0} = {1}",
                        attribute.Key, 
                        attValues[0]);
                    for (int i = 1; i < attValues.Length; i++)
                    {
                        Console.WriteLine("  {0} = {1}", 
                            new String(' ', attribute.Key.Length),
                            attValues[i]);
                    }
                }
                else
                {
                    Console.WriteLine("  {0} = {1}",
                        attribute.Key, attribute.Value);
                }
            }
            Console.WriteLine("End of ListServerAttributes");

            // ----- Start of section to replace -----
            var ous = new List<string>();

            ILdapPage<string> page = context.Query<OrganizationalUnitObject>()
                .Where(o => o.organizationalUnitName.StartsWith("CS"))
                .Select(o => o.distinguishedName)
                .ToPage(10);
            if (page.Count > 0)
                ous.AddRange(page.ToList());

            if (page.HasNextPage)
            {
                ILdapPage<string> nextPage = context.Query<OrganizationalUnitObject>()
                    .Where(o => o.organizationalUnitName.StartsWith("CS"))
                    .Select(o => o.distinguishedName)
                    .ToPage(page.PageSize, page.NextPage);
                if (nextPage.Count > 0)
                    ous.AddRange(nextPage.ToList());
            }

            foreach (var ou in ous)
            {
                counter++;
                Console.WriteLine("Ou: {0}",
                    ou); //.organizationalUnitName);
            }
            // ----- End of section to replace -----

            Console.WriteLine("Counter = {0}", counter);
            Console.WriteLine("\r\nPress a key to continue...");
            Console.ReadKey(true);
        }
    }

 

 

and here are the results:

 

Search Request >
Filter: (objectClass=*)
Attributes:
Naming Context:
Scope: Base
Types Only: False
Controls:
Page Size: 1
Page Cookie Length: 0
Page Control Is Critical: True
Page Control OID: 1.2.840.113556.1.4.319


ListServerAttributes
  supportedldappolicies = MaxPoolThreads
                        = MaxDatagramRecv
                        = MaxReceiveBuffer
                        = InitRecvTimeout
                        = MaxConnections
                        = MaxConnIdleTime
                        = MaxPageSize
                        = MaxQueryDuration
                        = MaxTempTableSize
                        = MaxResultSetSize
                        = MaxNotificationPerConn
                        = MaxValRange
  dnshostname = dc1.big.wooden.badger
  supportedldapversion = 3
                       = 2
  defaultnamingcontext = DC=big,DC=wooden,DC=badger
  schemanamingcontext = CN=Schema,CN=Configuration,DC=wooden,DC=badger
  supportedsaslmechanisms = GSSAPI
                          = GSS-SPNEGO
                          = EXTERNAL
                          = DIGEST-MD5
  issynchronized = TRUE
  subschemasubentry = CN=Aggregate,CN=Schema,CN=Configuration,DC=wooden,DC=badger
  highestcommittedusn = 192574042
  supportedcontrol = 1.2.840.113556.1.4.319
                   = 1.2.840.113556.1.4.801
                   = 1.2.840.113556.1.4.473
                   = 1.2.840.113556.1.4.528
                   = 1.2.840.113556.1.4.417
                   = 1.2.840.113556.1.4.619
                   = 1.2.840.113556.1.4.841
                   = 1.2.840.113556.1.4.529
                   = 1.2.840.113556.1.4.805
                   = 1.2.840.113556.1.4.521
                   = 1.2.840.113556.1.4.970
                   = 1.2.840.113556.1.4.1338
                   = 1.2.840.113556.1.4.474
                   = 1.2.840.113556.1.4.1339
                   = 1.2.840.113556.1.4.1340
                   = 1.2.840.113556.1.4.1413
                   = 2.16.840.1.113730.3.4.9
                   = 2.16.840.1.113730.3.4.10
                   = 1.2.840.113556.1.4.1504
                   = 1.2.840.113556.1.4.1852
                   = 1.2.840.113556.1.4.802
                   = 1.2.840.113556.1.4.1907
                   = 1.2.840.113556.1.4.1948
                   = 1.2.840.113556.1.4.1974
                   = 1.2.840.113556.1.4.1341
                   = 1.2.840.113556.1.4.2026
  domainfunctionality = 2
  configurationnamingcontext = CN=Configuration,DC=wooden,DC=badger
  domaincontrollerfunctionality = 3
  forestfunctionality = 2
  rootdomainnamingcontext = DC=wooden,DC=badger
  currenttime = 20120115083532.0Z
  isglobalcatalogready = TRUE
  servername = CN=dc1,CN=Servers,CN=Camelot,CN=Sites,CN=Configuration,DC=wooden,DC=badger
  supportedcapabilities = 1.2.840.113556.1.4.800
                        = 1.2.840.113556.1.4.1670
                        = 1.2.840.113556.1.4.1791
                        = 1.2.840.113556.1.4.1935
  ldapservicename = wooden.badger:dc1$@big.wooden.badger
  dsservicename = CN=NTDS Settings,CN=dc1,CN=Servers,CN=Camelot,CN=Sites,CN=Configuration,DC=wooden,DC=badger
  namingcontexts = CN=Configuration,DC=wooden,DC=badger
                 = CN=Schema,CN=Configuration,DC=wooden,DC=badger
                 = DC=big,DC=wooden,DC=badger
                 = DC=ForestDnsZones,DC=wooden,DC=badger
                 = DC=DomainDnsZones,DC=big,DC=wooden,DC=badger
End of ListServerAttributes
Search Request >
Filter: (&(objectCategory=organizationalUnit)(ou=CS*))
Attributes: distinguishedname
Naming Context: DC=big,DC=wooden,DC=badger
Scope: Subtree
Types Only: False
Controls:
Page Size: 10
Page Cookie Length: 0
Page Control Is Critical: True
Page Control OID: 1.2.840.113556.1.4.319


Ou: OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS,OU=VI SEC Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Access Groups,OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Business Planning,OU=Corporate,OU=VI TST Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Computers7,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS ComputersSPP,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Disabled Accounts,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS GPO Groups,OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Counter = 10

Press a key to continue...

 

 

Here's it modified to remove the paging code:

 

class Program
    {
        static void Main(string[] args)
        {
            var config = new LdapConfiguration()
                .AddMapping(new OrganizationalUnitObjectMap());

            config.ConfigureFactory("dc1");

            var counter = 0;
            var context = new DirectoryContext();
            context.Log = Console.Out;

            IEnumerable<KeyValuePair<string, object>> attributes = context.ListServerAttributes();
            Console.WriteLine("ListServerAttributes");
            foreach (var attribute in attributes)
            {
                if (attribute.Value is System.String[])
                {
                    var attValues = attribute.Value as System.String[];
                    Console.WriteLine("  {0} = {1}",
                        attribute.Key, 
                        attValues[0]);
                    for (int i = 1; i < attValues.Length; i++)
                    {
                        Console.WriteLine("  {0} = {1}", 
                            new String(' ', attribute.Key.Length),
                            attValues[i]);
                    }
                }
                else
                {
                    Console.WriteLine("  {0} = {1}",
                        attribute.Key, attribute.Value);
                }
            }
            Console.WriteLine("End of ListServerAttributes");

            // ----- Start of section to replace -----
            
            var ous = context.Query<OrganizationalUnitObject>()
                .Where(o => o.organizationalUnitName.StartsWith("CS"))
                .Select(o => o.distinguishedName);
            foreach (var ou in ous)
            {
                counter++;
                Console.WriteLine("Ou: {0}",
                    ou); 
            }
            // ----- End of section to replace -----

            Console.WriteLine("Counter = {0}", counter);
            Console.WriteLine("\r\nPress a key to continue...");
            Console.ReadKey(true);
        }
    }

 

 

and here are the results:

 

End of ListServerAttributes
Search Request >
Filter: (&(objectCategory=organizationalUnit)(ou=CS*))
Attributes: distinguishedname
Naming Context: DC=big,DC=wooden,DC=badger
Scope: Subtree
Types Only: False
Controls:
Page Size: 500
Page Cookie Length: 0
Page Control Is Critical: True
Page Control OID: 1.2.840.113556.1.4.319


Ou: OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS,OU=VI SEC Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Access Groups,OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Business Planning,OU=Corporate,OU=VI TST Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Computers7,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS ComputersSPP,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Disabled Accounts,OU=CS Computers,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS GPO Groups,OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Groups,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS IT Transformation,OU=Corporate,OU=VI TST Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CS Member Servers,OU=Div1 Member Servers,OU=Domain Member Servers,DC=big,DC=wooden,DC=badger
Ou: OU=CS Server Admin Groups,OU=Div1 Server Admin Groups,OU=Server Admin Groups,OU=Server Groups,OU=Centralised Accounts,DC=big,DC=wooden,DC=badger
Ou: OU=CS Users,OU=CS,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CSR,OU=Shared Data,OU=Home,OU=Buying,OU=QR,OU=VI SEC Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Ou: OU=CSR,OU=Shared Data,OU=Buying,OU=QR,OU=VI SEC Groups,OU=VI Groups,OU=VI,OU=DIv1,DC=big,DC=wooden,DC=badger
Ou: OU=CSV files,OU=P2W Budgeting,OU=VI-Mainframe,OU=Finance,OU=QR,OU=VI SEC Groups,OU=VI Groups,OU=VI,OU=Div1,DC=big,DC=wooden,DC=badger
Counter = 17

Press a key to continue...

 

 

Also, using InPagesOf, here's the source:

 

            // ----- Start of section to replace -----

            var ous = context.Query<OrganizationalUnitObject>().InPagesOf(10);

            foreach (var ou in ous)
            {
                counter++;
                Console.WriteLine("Ou: {0}",
                    ou.organizationalUnitName);
            }

            // ----- End of section to replace -----

 

 

and here are the results:

 

End of ListServerAttributes
Search Request >
Filter: (objectCategory=organizationalUnit)
Attributes: distinguishedname, ou, description, objectGuid, whenChanged, whenCreated
Naming Context: DC=big,DC=wooden,DC=badger
Scope: Subtree
Types Only: False
Controls:
Page Size: 10
Page Cookie Length: 0
Page Control Is Critical: True
Page Control OID: 1.2.840.113556.1.4.319


Ou: Domain Controllers
Ou: Div1 Domain Controllers
Ou: Div2 Domain Controllers
Ou: Div3 Domain Controllers
Ou: Domain Member Servers
Ou: Div3
Ou: Div3 All New Computers
Ou: Centralised Accounts
Ou: DataBase Server Accounts
Ou: Badger Administration
Counter = 10

Press a key to continue...

 

 

If I remove

.InPagesOf(10)

from that last approach, then I get 500 objects.  Hope that helps.  I've posted the whole project to my SkyDrive and emailed you a link.

Jan 20, 2012 at 5:11 AM

Can you try this and tell me if _nextPage has a length?

private byte[] _nextPage;
private int _pageSize = 10;
private void Page()
{    
    var control = new PageResultRequestControl(_pageSize) {Cookie = _nextPage};
 
    var request = new SearchRequest
                        {
                            DistinguishedName = "DC=big,DC=wooden,DC=badger",
                            Filter = "(&(objectCategory=organizationalUnit)(ou=CS*)",
                            Scope = SearchScope.Subtree
                        };
    request.Controls.Add(control);
    using (var connection = new LdapConnection("dc1"))
    {
        var response = connection.SendRequest(request) as SearchResponse;
        var next = response.Controls[0] as PageResultResponseControl;
        _nextPage = next.Cookie;
    }
}

Jan 26, 2012 at 10:25 AM
Edited Jan 26, 2012 at 9:27 PM

No, it doesn't: it's zero.  It's strange: response contains ten entries.  (There are 17, in total.  If I set _pageSize to 100, I see all 17 returned.)

I've done some investigating.  The queries can be quite slow: I would've expected them to be very fast, given the small page size, so that's odd in itself.  I've had to add connection.Timeout = new TimeSpan(0, 5, 0); to get them to complete.

I've pulled filter out as a parameter on Page(String filter), added a few Console.WriteLines, and then run it for different filters and this is what I saw:

Filter = (&(objectClass=user)(objectCategory=person))
No. of entries in response = 10
Cookie length = 524
Filter = (&(objectClass=computer)(objectCategory=computer))
No. of entries in response = 10
Cookie length = 524
Filter = (&(objectClass=group)(objectCategory=group))
No. of entries in response = 10
Cookie length = 0
Filter = (&(objectClass=organizationalUnit)(objectCategory=organizationalUnit))
No. of entries in response = 10
Cookie length = 0
Filter = (&(objectClass=msDS-AzOperation)(objectCategory=msDS-AzOperation))
No. of entries in response = 10
Cookie length = 0

Which is a bit odd, isn't it?

**** EDIT ****

I've been on this all day: my eyes seem to be stuck focussed 18" in front of me but I think I've solved this.  I haven't gotten to the exact cause but I have a solution; some more work will be needed to pin down the exact cause.

In summary, I looked at all sorts of angles.  Having seen the same problem in S.DS.P, I tried it on a different platform (2008 R2 instead of XP): same problem.  I used someone else's S.DS.P paging code which was essentially the same but formatted differently: same problem. 

Eventually, I tried searching in a specific OU and it worked, so I tried changing the NamingContext on my ClassMaps.  You'll recall that I tend to use the defaultNamingContext of the domain.  I set it to be a specific OU with only a few objects and the search worked.  Not only that but it ran qucikly - this should've been a clue but I missed it.  I tried different OUs, nearer to the root and they all worked.  It was only when I got to the dNC that they took ages and stopped after one page.  I tried this with different objectClasses and got the same results.  NamingContext = dNC => failure and long searches.

In searching for an S.DS.P forum where I could ask a question about this, I happened on this post, which contains the solution.  To your suggested code, I added:

    var soc = new SearchOptionsControl(SearchOption.DomainScope);
    request.Controls.Add(soc);

with the equivalent addition to my original code being:

    .WithControls(new [] { new SearchOptionsControl(SearchOption.DomainScope) } )

and that solved it.  All of the expected results are returned, and in a timely fashion.  I haven't gotten to the bottom of why this is needed: I'm too frazzled to read anymore.  That's a subject for another day.  I'm just glad it works, now.

Thanks for your suggestion - it put the on the right track.