Dreamforce

Dreamforce this year was a fantastic experience.

  • I came home with two certifications, Platform Developer I and Administrator.
  • My talk with Adam Kramer, Staying on Top of Salesforce & NPSP Releases, had a nice reception, and I got to chat with some excellent nonprofit Salesforce users.
  • I learned a ton of new techniques for writing generic and dynamic Apex code, using mocks and stubs to build better unit tests, implementing continuous integration, and practicing modern development on the Salesforce platform.

I can’t wait to go back.

A year of officiating

I’m very happy to have been elected to another year as Head of Officials for my roller derby league, Duke City Roller Derby. I think we’ve made a lot of progress in 2016 and I’m grateful that DCRD took a chance on me, then a very new NSO, last year.

It’s been a busy and exhilarating year. I’ve officiated 51 games in six states, participated in 3 tournaments, traveled (by a back-of-the-envelope reckoning) something on the order of 10,000 miles, met tons of fantastic people, and had a great time. My personal goal for 2017 is to join at least two higher-level tournaments and aim for WFTDA certification.

Dreamforce

I’m really excited to be speaking next week at Dreamforce 2016 — my first trip to the conference. If you’re interested in Nonprofit Success Pack maintenance and updates, come watch my session! Staying on Top of Salesforce & NPSP Releases starts at 10 am on Tuesday, October 4.

Importing Notes in Salesforce

Building Note records in Salesforce, using the new notes introduced in Winter ‘16, is challenging for a number of reasons. They come with unusual data-preparation requirements and are tricky to import using the Data Loader. Notes are tricky to create in Apex for largely the same reasons. Failing to follow the requirements for encoding incoming data typically produces notes with all of the line breaks omitted and/or unpredictable and difficult-to-debug exceptions.

To make matters worse, various documentation entries are either incorrect (the API reference on ContentNote specifies to use String.escapeHTML4() to prepare content, which doesn’t work) or overly vague and/or incomplete (the String API documentation on escapeHTML4() and escapeXML(), the ContentNote import instructions).

In summary, here’s what is actually required to insert ContentNotes:

  1. Replace all basic HTML characters (<>"'&) with their corresponding entities (&amp; and friends).
  2. Replace all line breaks with <br> (taking care with Windows CRLF/Linux LF/Mac CR)
  3. Replace &apos; with &#39;.
  4. Do not replace Unicode characters with entities. Other entities, including &apos;, result in an exception. Unicode should be left as the bare characters.
  5. Ensure that the source content is well-formed Unicode/UTF-8 and does not contain non-printable characters.
  6. The title must not be null, zero-length, or consist only of whitespace. The title need not be escaped.

This 4th point is why String.escapeHTML4() doesn’t work for preparing note text: this method replaces Unicode characters with HTML entities, which causes exceptions upon insert. String.escapeXML() is closest to what is needed, but doesn’t handle item 3 above.

In frustration with all the hoops involved, I put together a package that can reliably import and add notes and attachments, both programmatically and in bulk. The package provides a DMRNoteAttachmentImporter class that can be used in Apex code to create note and attachment records either singly or in bulk, as well as a Note Proxy object that can be converted into Notes using a batch process. Notes may be imported to Note Proxy from a CSV file using any data loader (obviating the one-file-per-note requirement imposed by Apex Data Loader).

Attachments are handled alongside Notes, since the machinery for creating them is very similar.

The code is MIT licensed. It will fail if the new notes have not been enabled in your Salesforce instance. It’s been tested with both Apex unit tests (100% coverage) and example files. Bug reports and patches are welcomed.

Be aware that, using the Note Proxy object, you can only import notes of lengths up to the limit of a Long Text Area, 131,072 characters (128KB, assuming 1-byte characters). In my testing, it’s fine to process 32KB notes in batches of 200. Stepping up to 64KB notes (which some spreadsheet applications and data loaders cannot process in CSV format in any case) required a reduction in batch size.

Errors that do occur with ContentNotes usually come in the form of System.UnexpectedExceptions, which cannot be caught or handled. Despite the best efforts of this package, it is still possible to trigger these exceptions by attempting to import notes whose text contains non-printable characters or mangled UTF-8. In a bulk note import process, this will cause the failure of the entire batch with no error message recorded on the note proxies. The error can be diagnosed by examining the Apex Jobs log, where the exception will be displayed. The only workaround is to use very small (or even 1) batch sizes to identify the offending note and manually correct its text.

Resources

Private Email Lists using MailChimp and Salesforce

One of the biggest challenges I’ve encountered in working with MailChimp and its MailChimp for Salesforce application is ‘private email lists’. I mean by this email lists that are designed for communication with, for example, major donors, board members, or other small groups of contacts that are defined by established relationships and specific CRM criteria. These groups may change composition regularly as new members are added or as old members drop out of the matching criteria, and no public user interface is required other than the standard opt-out.

MailChimp for Salesforce’s overall integration model is additive. MC Queries in Salesforce identify groups of records that must be included in a mailing list and pass those records into MailChimp, where any new records are appended. The reverse, however, is not the case. If a record ceases to match an MC Query, it is not removed from its corresponding list or groups. MC Queries are also fairly limited in the logic upon which they can match: criteria may only be combined by AND.

For many private email lists, this won’t work: donors stop giving and members rotate into and out of various internal groups with equal alacrity. Updating MailChimp private groups by hand is both error-prone and time-consuming. Here’s the solution I worked out to support these private email lists natively in MailChimp without manual maintenance.

  1. Use formula fields in Salesforce to implement inclusion logic. For example, a courtesy invitation list for donors might be defined as those with a “VIP” checkbox selected or giving in excess of $1,000 either this year or last year. Those three criteria could not be represented in a MailChimp for Salesforce MC Query, which only allows AND logic, but a Salesforce formula field with type Checkbox easily encompasses them.
    • MailChimp’s segment definition logic (see below) does allow for the use of OR-based logic. If you have many lists that are defined exclusively by ORing together different subsets of the same relatively small set of criteria, you may be able to skip this step.
  2. Create a new list in MailChimp to hold the private email lists. (It’s possible to use an existing publication list, but I felt that these communications were different in kind and that a separate list would provide a better user experience). I called mine “Private Announcements and Invitations”.
    • If existing lists are mailed to as a whole (as opposed to groups or segments within the list), you’ll definitely want to create a new list. Your Private Announcements list shouldn’t be mailed to as a whole, since members will remain in the list even when they stop matching your segments.
  3. In MC Setup in Salesforce, ensure that the formula fields (or the basic criteria if you’re implementing the logic in the MailChimp segments) you created to support list logic are mapped to MailChimp for this new list. Mapped fields, unlike list membership per se, are re-synced to MailChimp in a regular process, and Salesforce updates are reflected in the mapped data.
  4. In MailChimp’s settings interface, ensure that all of these fields are not marked Visible. This prevents users from viewing or editing these internal values by accessing MailChimp’s profile update UI.
  5. Build MC Queries to match each of the list-definition formula fields and set them to run on a schedule. The queries don’t need to be (and shouldn’t be) associated with a group or segment.
  6. Use MailChimp’s segment builder to create a segment for each private email list, matching on the value “true” in the mapped Salesforce formula field. (My formulas are all checkboxes, but they don’t need to be).
    • If you opted to use MailChimp logic rather than formula fields, configure your criteria here.
  7. Save each segment as an Auto-Update Segment. Auto-Update Segments do remove members who cease to match their defined criteria.

The end result is a MailChimp list with one Auto-Update Segment per private email list, all of which maintain themselves based on the membership information that is calculated by the formula fields and synced from Salesforce. List members who fail to match criteria do remain members of the containing MailChimp List, but will fall out of segments (the actual private email lists) to which they no longer belong as soon as the containing list syncs.

The time resolution for list updates is between one and 24 hours. MailChimp syncs lists from Salesforce on an hourly basis, which updates all of the synchronized fields. However, the MC Queries which add new members to the lists only run every 24 hours. Hence, you can count on updated information, including changed criteria that will prompt a member to drop off an Auto-Update Segment, to propagate to MailChimp within one hour, but contacts who match list criteria for the first time may not be added for up to 24 hours. Fortunately, you can always force a list sync or a query run from the MC Settings tab in Salesforce.