Sharepoint 2013, the not quite getting it release

‘…Your Enterprise Social Network…’ well almost.

Compared to 2011, 2013 is clearly a large step forward. Document editing works and in my opinion is superior to the google docs alternative, the layout is much, much easier to navigate and at long last MS seems to have backtracked on their Document Libraries are not like shared folders stance. The Document library to Explorer integration and connection to office overall is much improved (read, it works).

With all the good things, there appear to be a few glaring omissions. Given the glaring holes, it appears that the Sharepoint dev group either does not eat their own dog food, or has a very convoluted day to day work process.

1. Following and forgetting.

You can follow sites and documents, but not folders. We’ve come across a number of times when sharing part of a document library (ie, working on a single presentation, collecting a limited set of documents, sharing small groups of ppts/word documents) is necessary. This works nice, click share type in your colleague’s name, then for the love of god, make sure your colleague doesn’t misplace that e-mail. There’s no way to follow that folder. I can follow every single document which is good until someone uploads another document. If its not a top level folder, browsing to that document library doesn’t show it :(.

2. Folders and a filesytem, but not really

Sharepoint 2010, don’t use folders and document libraries as file shares, in Sharepoint 2013 you mount a document library locally but folders are bad, use tags instead. NO! If you attach it to Explorer, it should behave like a filesystem, people will use it like a filesystem. Don’t give me a convenient way to access stuff, then say no.

Pretty much all the caveats about file size, characters, path length listed in the Migrating File Shares to Skydrive Pro blog post mean that Skydrive Pro is pretty useless for all but the most simple cases.

3. Finding s^$%

You know what’s nice, when someone shares something with you, or gives you access to something, not having to make a conscious effort to bookmark or follow it. Again, take a cue from Dropbox, google docs, if someone shares something, grants me or my group access it, call me crazy, but I probably want to easily access it.  If I don’t, then leave the onus of removing it on me.

A possible solution, allow sharing to automatically add stuff to My Sites, or a shared document library. Don’t make me use search to find stuff that should be one click away.

4. Sharing, but not to everyone

Apparently MS only e-mails documents to folks outside of Redmond or only shares documents with people within their corporate borders. Box, Dropbox, Google docs, Pydio, and well pretty much everyone else lets me e-mail an obfuscated link to a document that will (shocker!) open that document or folder. Is it perfectly secure? No, however it definitely falls under the lets me get work done category.

And before you start, forcing my colleagues to get an outlook account and federating to outlook.com is not an acceptable solution (here’s looking at you Skype/Lync).

5. Its not a Windows world anymore.

Skydrive Pro and Sharepoint document syncing is wonderful… I love it, its corporate Dropbox for my office documents finally… And its a complete pain to support anyone that’s not running a PC attached to my domain. Telling your OS-X users to use the web to download/open documents isn’t a solution. Having to use the web browser on Android to retrieve documents, not a solution. If you’re not on Windows, you’re a second class citizen isn’t a solution. People have seen the future of document syncing and it looks and behaves like Box and Dropbox, please copy it.

On a related note: Box/Dropbox, give me an on-premise solution for sharing and you’ll probably end up giving Sharepoint a run for its money.

All in all, SP 2013 is a huge leap forward, document editing in office web apps is light years ahead of google docs, navigation, the overall layout, site templates, etc are incredibly powerful. Its a shame that MS is still determined to do things their way, as opposed to what is in the best interest of their customers.

Lync and Updates and 80240437

For future reference, disabling TLS 1.2 on your Lync 2013  standard edition frontend will break your ability to update windows server 2012. You’ll end up w/ error 80240437 that’s pretty damn useless. Meanwhile, manual installation of cumulative updates work, and your other (mediation, edge, monitor, etc) roles all work and patch fine, just your frontend  is f’d. Hopefully you stumble across this technet post and look at the second to last post before you go down the rabbit hole. A light bulb will go off and you’ll remember the registry hacks you applied from here several months ago. To patch your server, you’ll need to disable the reg hacks by re-enabling TLS 1.2 (set disable to 0), running windows update, then re-enable the registry hacks. Afterwards, grab some rye and curse Microsoft.

 

More Lync Meeting testing

Just ran a mostly successful Lync 2013 meeting with folks that do quite a bit of video/teleconferencing already.

Some observations on how the meeting went:

  • 1 participant joined via phone
  • 1 via Windows 7, firefox and the web client, no problems
  • 1 via OS-X, firefox and the web client, no problems
  • 1 joined via Win8, 64bit and office 2013. Client would freeze when starting. Luckily there was a backup computer with similar specifications that did work
  • 1 Win 7, office 2013 Lync client. Client would periodically slow down, audio/video stuttering, CPU load spiking at 100% (4 core, 8g ram!) then freeze. After 30s client would wake up and continue working with no problem.

Overall quality was significantly better than Adobe Connect, however the client issues were very disappointing. Hopefully they were one-off errors, however my hopes are fading that 2013 is a viable Adobe Connect replacement.

Building a (Relatively) Affordable Conference Room

(Hardware updated 10/2016)

Our standard Adobe Connect, Skype and Lync compatible conference configuration is designed to provide skype-quality audio and video in and out of a conference room. While a more advanced, high end system would be nice, the types of hardware/software that our ever changing endpoints make that prohibitively expensive. Instead, we rely on most current software’s ability to provide decent full duplex built-in echo cancellation. Flash, Skype and Lync all do this pretty well, we’ve had some difficulty with Webex and early releases of Google Hangouts.

Our goal is to provide complete audio coverage for any participant sitting at a table in our conference room. As our meetings tend to be mostly round-table style discussion a rule of one microphone for every two people allows us to pick up normal conversation-level speech.

Our requirements are that we allow remote participants virtually join meeting in conference rooms ranging in size from 8 through 24 people. Realistically for groups larger than 16-18 the logistics of ensuring that remote participants are fully included in a meeting starts to break down. Distance to TV, etc start to have an detrimental effect on the ability of remote participants to be heavily engaged in a meeting.

Video:

Audio:

Television/Stand:

Misc Parts:

  • Velcro straps and carpet cover
  • Under-table clips for microphone cables
  • wire wrap

Total Cost (no PC): ~$3,000 $3,700 (8 person) – $5,150 $5,850(24 person)

Lync 2013 or Adobe Connect

At our shop, many of our working group meetings routinely (over 70%) have remote participants that attend for all or part of their meeting. Unfortunately the platforms and connections these remote participants have access to varies widely. We’ve had participants from every continent except Antarctica, with access to any number of 3g/4g, high speed research network, DSL and other home access using all i* and win* devices available. Due to the ever changing remote endpoints this has ruled out your more traditional high end room system environments.

Our first generation room setup was based on a cloud-managed Adobe Connect reseller or Skype depending on how the meeting would be run. Skype for point to point/managed meetings and Adobe Connect for multi-person or if flexibility for leave/join of remote participants was required.

A few things that worked surprisingly well in this setup:

  • Clients everywhere. The native Adobe Connect and Skype clients provided connectivity for almost every (not linux) platform under the sun. With native ios and Android clients we’ve seen an increase in tablet guests.
  • All conferences are backed by an 8xx dial-number in case of audio or phone-only connections. For an 8 hour meeting, total cost was only ~$10 to keep a line open.
  • Little to no low level driver/OS troubleshooting required. Devices either worked or they didn’t and the client performed the same regardless of what was attached.
  • Ease of use. Remote guests require around 2-5m of brief hand-holding to become familiar with the client, and connection steps are straight forward.
  • Flash echo-cancellation works pretty well.

A few things that we found that would have been nicer to have:

  • No higher quality video, or customizable video layouts. Brady-bunch square or nothing.
  • No room-system integration. We’ve had to use the same commodity hardware in our rooms that end users did. Single camera, etc.
  • Large latency impacts. Attempts to do video across large distances introduced noticeable lag on the part of the remote participants.
  • Flash echo cancellation is not enabled by default.

Conferencing Rev 2

We’ve recently upgraded our on-site Lync installation to 2013 in hopes that it will offer a viable alternative to Adobe Connect for remote meetings. The ability of Lync 2010 to support multi-party meetings with external parties was unworkable almost to the point of being a liability. (Windows-only meeting clients, half working OS-X, silverlight…). Even assuming all that worked, the lack of a true meeting screen limited communications. (no multi-camera, etc)

Now 2013 appears to be very promising  and looks like it may correct many of the shortcomings of its previous version.

We’ve started preliminary testing and deployment and have found a few nice additions.

  • Integration with real room systems. Purchasing an out of the box conference room is now an option whereas in the past it was not.
  • HD video.
  • Superior echo cancellation. Even in 2010, the built-in aec appeared to be superior to the native flash client.
  • Web client appears to work all major Win and OS-X platforms and browsers.
  • Multi-monitor native clients when you separate the video window.
  • Direct Power Point connection for Office 2013.

Now a few drawbacks:

  • Client unpredictability (both native and web). We’ve seen instances where the client will freeze for periods of time, peg the CPU and behave differently based on attached hardware. Running through this matrix with remote participants promises to be a nightmare.
  • Tablets. The tablet client isn’t a meeting-only client, but rather designed to be hooked up to an existing Lync account.
  • Skype Integration. Audio only (I know video is forthcoming) and the requirement that remote parties link their Skype account to an outlook.com or other MS makes this unworkable. You try explaining the subtle sign-in difference to a senior scientist who can’t talk to their non-MS-enabled colleague.

So Lync or Adobe Connect?

Will we use it in the long run? I hope so. In the meantime, a few areas of testing remain including finding and testing on low-resource clients and quantifying high latency behaviour. Our biggest area of concern is the observed unpredictability of the client. Given the wide variety of hardware and OS’s we encounter if the clients continue to behave unpredictably, the maintenance and troubleshooting overhead will make Lync not worth the effort. With Adobe Connect as long as flash was updated unpredictability generally was not a problem and the client behaved quite reasonably.

Sympa and Active Directory

Some basic steps on running sympa on Ubuntu 12.04 and using Active Directories Global Directory to auto-populate groups.

Ubuntu Notes 

  • apt-get install sympa will give you a ‘mostly’ working version
  •  Chown -R /var/lib/sympa sympa
  • The suid wrapper does not work on 12.04. You will need to create a sudo wrapper instead:
  • set use_fast_cgi 1 in /etc/sympa/wwsympa.conf
  • /usr/lib/cgi-bin/sympa/wwsympa_sudo_wrapper.fcgi
  • #!/usr/bin/perl
    
    exec '/usr/bin/sudo', '-E', '-u', 'sympa', '/usr/lib/cgi-bin/sympa/wwsympa.fcgi';
  • In apache/conf.d/sympa, change:
    ScriptAlias /wws /usr/lib/cgi-bin/sympa/wwsympa_sudo_wrapper.pl
  • add the following line to your sudoers file:
    www-data ALL = (sympa) SETENV: NOPASSWD: /usr/lib/cgi-bin/sympa/wwsympa.fcgi
  • References:

LDAP/AD Bound Lists

  • If you only have one domain, then you can just use the following and point at one of your domain controllers.
  • If you want to use forest-wide groups, you have two options for accessing those groups.
  • This will work this either security or distribution groups, however will NOT include nested membership.
    • In the ldap config for the group, point at the dc the group resides in. CHange suffix, host and user as appropriate, set use_ssl to yes, drop the :3268
    • Make the group universal and use the global directory (route I chose)
  • LDAP Configuration
    include_ldap_query
    attrs mail
    filter memberof=Some Group,OU=...,OU=...,DC=research,DC=domain,DC=org
    ssl_ciphers ALL
    name any_name
    host dc1.mydomain.org:3268
    use_ssl no
    passwd your_password
    timeout 30
    suffix DC=domain,DC=org
    user   CN=Read Account,OU=...,DC=domain,DC=org
    ssl_version sslv2
    scope sub
    select first
    ssl_version tls
  • References

Fun with DC’s gis data, part 1

It looks like DC has kindly released quite a bit of gis data for public consumption. One of the more interesting sets is the regularly updated Owner Polygon dataset available from data.dc.gov. This is a shapefile containing current property records for everything in the District. Unfortunately, it’s not available kml for easy display in google’s tools. However the 70MB esri shapefile is available. Using Open Layers, PostGIS, and and GeoServer, we can get start displaying everything, but what if we want to use google maps and do things the hard way?

To solve that, there’s a few simple steps to allow polygon querying, selection, and display on google maps.

  1. Import data into PostGIS
  2. Create GIS servlet
  3. Draw the data on google maps
  4. Query PostGIS for google’s lat/long
  5. Select Properties from the map

We’re going to work on step one today, import your data into PostGIS.

Prepare PostGIS

I’m running Ubuntu 11.04, PostgreSQL 8.4 with postGIS 1.5.1 installed from the default software repo.

  1. PostGIS 1.5 manual
  2. nad 83, maryland projection
psql (8.4.8)
Type "help" for help.

postgres=# create database propertymap;
CREATE DATABASE
postgres=# \q
~$ createlang plpgsql propertymap;
~$ cd /usr/share/postgresql/8.4/contrib/postgis-1.5/
postgis-1.5$ psql -f postgis.sql propertymap
postgis-1.5$ psql -f ../postgis_comments.sql propertymap;
postgis-1.5$ psql -f spatial_ref_sys.sql propertymap;

Convert Shapefile

Create a ton of insert statements using shp2pgsql:

poly$ shp2pgsql -s 926985 OwnerPly.shp ownertable > inserts.sql
Shapefile type: Polygon
Postgis type: MULTIPOLYGON[2]

If we look at the .prj file included, we see that the projection for the data is NAD_1983_StatePlane_Maryland_FIPS_1900. We need to add the projection from spatialreference.org in to our database

propertymap=# INSERT into spatial_ref_sys (srid, auth_name, .......66666666],UNIT["Meter",1.0]]');
INSERT 0 1
propertymap=# \i inserts.sql

Run your first query

propertymap=# select ownername,square,lot,premiseadd from ownertable where premiseadd like '%1600 PENNSYLVANIA%';
        ownername         | square | lot  |       premiseadd        
--------------------------+--------+------+-------------------------
 UNITED STATES OF AMERICA | 0187   | 0800 | 1600 PENNSYLVANIA AV NW
 UNITED STATES OF AMERICA | 0187   | 0802 | 1600 PENNSYLVANIA AV NW
 UNITED STATES OF AMERICA | 0187   | 0801 | 1600 PENNSYLVANIA AV NW

First, a little background on what we asked for. DC property records are based on square, suffix, and lot. Square generally refers to a city block and goes all the way back to the original city planning in the old part of the city. Lot is a lot within a square/suffix. For the most part, you can ignore suffix as it’s rarely used.

Next time, create a simple servlet to expose all of this.

Isolating Big Blue Button Video

This is a quick how to on manually connecting to a BBB video stream. Before we begin, here’s a very, very quick background.

  • Video streams are grouped under a conference-room specific url that has for format rtmp://host/video/roomID
  • Each streaming component under BBB is available as a separate stream (ie, video, desktop, sip/audio, etc)
  • BBB uses red5 under the hood to manage these streams
  • Grab flowplayer here and the flowplayer rtmp client here
  1. Connect to your room and start your webcam.
  2. Tail /usr/share/red5/log/bigbluebutton.log and uou should see the following log lines:
    2011-07-11 18:14:54,871 [NioProcessor-1] DEBUG o.b.c.s.p.ParticipantsEventRecorder - A participant's status has changed 141 streamName 640x480141
    2011-07-11 18:14:54,919 [NioProcessor-1] DEBUG o.b.c.s.p.ParticipantsService - Setting participant status ec0449a0-b5d1-4ca5-bfdf-d118d8bc2299 141 hasStream true
    • ec0449a0-b5d1-4ca5-bfdf-d118d8bc2299 or similar is the room id
    • 640×480141 is the stream id you need
  3. Download and place flowplayer-…swf, flowplayer.rtmp-…swf, and flowplayer-…min.js into a directory.
  4. Create a web page as follows:
  5. 
           
           Minimal Flowplayer setup
    
    
    
  6. Load up your web page and you should see the streaming video.

log4j and Pivot

Here’s a simple way to consume log4j messages in pivot for use in a log console or similar.

First create a custom appender which sents a log message to the pivot message bus.

public class MessageBusAppender extends AppenderSkeleton {
    
    @Override
    protected void append(LoggingEvent event) {
        MessageBus.sendMessage(new LogMessage(layout, event));
    }

    @Override
    public boolean requiresLayout() {
        return true;
    }

    @Override
    public void close() {
       //nop
    }

}

public class LogMessage {

    private Layout logLayout;
    private LoggingEvent event;

    public LoggingEvent getEvent() {
        return event;
    }

    public Layout getLogLayout() {
        return logLayout;
    }

    public LogMessage(Layout logLayout, LoggingEvent event) {
        this.logLayout = logLayout;
        this.event = event;
    }
    
}

In any component needs to display log messages, just listen for the messages and update as appropriately. Here’s an example updating a textpane:

public class LogPane extends Border {

    @BXML
    private TextPane logTxt;
    @BXML
    private PushButton clearBtn;
...
...
        logTxt.setDocument(new Document());

        MessageBus.subscribe(LogMessage.class, new MessageBusListener() {

            public void messageSent(final LogMessage message) {
                ApplicationContext.queueCallback(new Runnable() {

                    @Override
                    public void run() {
                        String text = message.getLogLayout().format(message.getEvent());
                        logTxt.getDocument().add(new Paragraph(text));
                        if (message.getEvent().getThrowableInformation() != null)
                        {
                            StringBuilder sb = new StringBuilder();
                            for (String s : message.getEvent().getThrowableInformation().getThrowableStrRep())
                            {
                                sb.append("  ");
                                sb.append(s);
                                sb.append("\n");
                            }
                            logTxt.getDocument().add(new Paragraph(sb.toString()));
                        }
                    }
                });
            }
        });

        clearBtn.getButtonPressListeners().add(new ButtonPressListener() {

            public void buttonPressed(Button button) {
                logTxt.setDocument(new Document());

            }
        });
 ...
 ...
}

Now tie it together in your log4j config:

log4j.rootLogger=ERROR, Pivot

log4j.appender.Pivot=sample.MessageBusAppender
log4j.appender.Pivot.layout=org.apache.log4j.PatternLayout
log4j.appender.Pivot.layout.ConversionPattern=%-6r [%15.15t] %-5p %30.30c %x - %m%n