How we launched 2GIS under CarPlay and are still unraveling

How we launched 2GIS under CarPlay and are still unraveling



Hello! My name is Vanya, I am writing a 2GIS mobile application for iOS. Today there will be a story about how our navigator appeared in CarPlay. I'll tell you how we created a work product with such documentation and unfinished tools and placed it in the AppStore.


A couple words about CarPlay



First, some materials to understand some aspects of the work of CarPlay and the reasons why we made certain decisions.


CarPlay is not an OS inside another OS, as it is written about in very many articles. If roughly, then CarPlay is a protocol for working with an external display of the screen of the head unit; sound from car speakers; touch screens, touch panels, washers and other input devices.


That is, the entire executable code is directly in the main application (not even in a separate extension!) This is very cool: to get new features, you do not need to update the radio tape recorder or even the car, you just need to update iOS.

< br/>

At WWDC 2018, Keynote was presented with the opportunity to create navigation applications for CarPlay, which made us very happy. Immediately after the presentation, we sent a request to authorize development for CarPlay. In the request it was necessary to show that our application can navigate.


While we were waiting for a response from Apple, a lecture was released, featuring the example of the CountryRoads sample application talked about working with CarPlay.framework. The lecture did not tell about the pitfalls and intricacies when working with CarPlay, but they mentioned that after connecting to the CarPlay radio tape recorder, the application will run in background mode.


The first stick in the wheel


The application’s work in the background’s has disappointed us. There were two reasons for this:


  1. We do not work in the background. Once left this limitation for technical reasons and for the sake of energy saving.
  2. Our map is written in OpenGL (yes, deprecated, yes, not Metal, we all know that), and OpenGL does not work in the background state. At best, you get a black view, and at worst, crash.

It was still possible to cope with the work in the background, but it was definitely necessary to solve something with the map. That's when the idea came to make it through the standard MKMapView. Until you started throwing stones at us for the idea of ​​using standard Apple maps, I'll explain: we were going to use MKMapView, but not Apple maps.


The fact is that MKMapView can load third-party tiles. Tiles are special rectangular containers for textures. We just had a servachok who knows how to give tiles. On GitHub there is a code with implementation.


Apple’s reply


We received a response from Apple, in which, in addition to the development permit, we also received the documentation for the elect, the CountryRoads sample code (it was shown at the WWDC lecture) and, most importantly, the private capability-key com.apple.developer.carplay-maps . This key is set in the entitlements file with a value of YES, so that the system understands that you can handle events from CarPlay when you start your application.


Without waiting for the sprint with the allocated for the development of history, I climbed to download Xcode Beta. The first attempt to collect 2GIS was a failure. But the draft sample application CoutryRoads managed to build a simulator.


Before each opening of the CarPlay window, the latter had to be customized through this window:



To do this, it was necessary to set the following line in the terminal: defaults write com.apple.iphonesimulator CarPlayExtraOptions -bool YES


For some reason, this did not work - I had to run almost on the smallest simulator with a resolution of 800 × 480 points and a scale × 2. At the moment, this setting works and helps a lot.


Having created my sample project and armed with documentation, I began to figure out what was happening.
The first thing I realized: navigation applications for CarPlay consist of base view and templates layers.



Base view is your map. On this layer there should be only a map, no other views and controls.


Templates is almost a non-customizable mandatory set of UI elements for displaying routes, maneuvers, lists of all sorts, and so on.


Beta Development


Let's go to the writing code. The first thing to do is to implement a couple of required CPApplicationDelegate methods in the ApplicationDelegate file.


  func application (
  _ application: UIApplication,
  didConnectCarInterfaceController controller: CPInterfaceController,
  to window: CPWindow
 ) {}

 func application (
  _ application: UIApplication,
  didDisconnectCarInterfaceController controller: CPInterfaceController,
  from window: CPWindow
 ) {}  

Let's look at the signature:


Everything is clear with UIApplication.
CPWindow - the heir of UIWindow, the window for the external display of the head unit radio.
CPInterfaceController is something like a UINavigationController’s analogue, only from CarPlay.framework.


We now turn directly to the implementation of the method.


  func application (
  _ application: UIApplication,
  didConnectCarInterfaceController controller: CPInterfaceController,
  to window: CPWindow
 ) {
  let carMapViewController = CarMapViewController (
  interfaceController: controller
  )
  let navigationController = UINavigationController (
  rootViewController: carMapViewController
  )
  window.rootViewController = navigationController
 }  

In didConnect, you need to write code similar to the one we used to see in didFinishLaunching. CarMapViewController is a base view (controller actually, but ok), as per documentation.


This is the final picture I received:



Somewhere around this time it dawned on me that in the new Xcode the new build system is enabled by default and, most likely, because of this, 2GIS is not going to.


I opened Xcode, put legacy (or rather stable, let's call things by their proper names) build system, and my theory was confirmed: 2GIS gathered.


Having set the same capability-key, I launched 2GIS under CarPlay and did not see any logs about the application switching to background mode. It became even more incomprehensible, because Apple engineers from the scene said about the background-mode, but, on the other hand, we were promised a contentView from UIAlertView, and as a result, UIAlertView became deprecated.


Deciding that it should be so, I did not bother with MKMapView. She would have deprived us of offline and forced us to re-write route drawing.


The problem of a single card


I didn’t have time to rejoice at the news that there would be our map in CarPlay, as I faced the following problem: because of the technical features, there can be only one card.
The quick solution to this problem was, though not very elegant.


Usually at the moment of using 2GIS on CarPlay, the phone is locked and lies somewhere on the shelf. This means that the card is not very necessary on the phone at this moment (it does not hurt to search, of course).Therefore, we decided to take the card from the main application when connecting the phone to CarPlay and display it on the CarPlay radio screen. And when disconnected, respectively, return back to the application on the phone.


Yes, the solution is such, but it is fast, it still works and did not have to kick a couple of other teams to rivet the MVP.


map controls


So, we got our map on the radio screen. Now it was necessary to make the first and obvious things for any map: controls of zoom, current location and map movement.



Let's start with the zoom and the current location, because these controls are on the map itself and these are not ordinary UIControl. As I wrote above, only the map is located on the base view.


In order to put these controls on the map, I had to climb into the documentation and sample application again. There I read about the first template - CPMapTemplate.



CPMapTemplate - a transparent template for displaying some controls on the map and an analog navigationBar. It is created and exhibited as follows:


  let mapTemplate = CPMapTemplate ()
 self.interfaceController.setRootTemplate (mapTemplate, animated: false)  

Next, you need to create these controls and put them on the map.


  let zoomInButton = CPMapButton (...)
 let zoomOutButton = CPMapButton (...)
 let myLocationButton = CPMapButton (...)

 self.mapTemplate.mapButtons = [
  zoomInButton
  zoomOutButton
  myLocationButton
 ]  

But the mapButtons array turned out to be a joke, because how many items you need in it, it will take only the first three items and display them on the screen. You won't get any errors in the log or asserts.


Then I climbed up to look at how to make a map move, and I found this in the documentation:


  There is no support for the user.   

Strange, I thought, and it’s helpful to look at how this is done in the CountryRoads sample application. The answer is through this interface:



Not very convenient, but in any other way, the documentation will not lie, right?


Since the place for controls on the map was over, it was necessary to make a button to switch the map to the “dragging” mode in this navigationBar’s analogue.


  let panButton = CPBarButton (...)
 self.mapTemplate.leadingNavigationBarButtons = [panButton]
 self.mapTemplate.trailingNavigationBarButtons = []  

But the arrays of the leadingNavigationBarButtons and trailingNavigationBarButtons were also not without a joke: how many elements in them nor push, they take only the first two. Also without errors in the log and asserts.


And to activate and deactivate the mode of dragging the card, you must write:


  self.mapTemplate.showPanningInterface (animated: true)
 self.mapTemplate.dismissPanningInterface (animated: true)  

Build and display routes on the map


Next, I proceeded to reuse our existing API to build routes.


Just for the demo and understanding what to do, I decided to take two points and build a route between them. Point A was the user's location, and Point B was our head office in Novosibirsk.


Code
  let choice0 = CPRouteChoice (
  summaryVariants: ["46 km"],
  additionalInformationVariants: ["including traffic jams"],
  selectionSummaryVariants: ["1 hour 7 min"]
 )
 let choice1 = CPRouteChoice (
  summaryVariants: ["46 km"],
  additionalInformationVariants: ["including traffic jams"],
  selectionSummaryVariants: [“1 hour 11 min"]
 )

 let startItem = MKMapItem (...)
 let endItem = MKMapItem (...)
 endItem.name = "Tolmachevo, international airport”

 let trip = CPTrip (
  origin: startItem,
  destination: endItem,
  routeChoices: [choice0, choice1]
 )

 let tripPreviewTextConfiguration = CPTripPreviewTextConfiguration (
  startButtonTitle: “On the road”,
  additionalRoutesButtonTitle: “More”,
  overviewButtonTitle: "Back"
 )

 self.mapTemplate.showTripPreviews (
  [trip],
  textConfiguration: tripPreviewTextConfiguration
 )  

On the screen, we received a control with a description of the route:



Navigation Mode


Routes are good, but the main feature of the navigator is still in navigation. To make it appear, you need to write the following:


  func mapTemplate (
  _ mapTemplate: CPMapTemplate,
  startedTrip trip: CPTrip,
  using routeChoice: CPRouteChoice
 ) {
  self.navigationSession =
  self.mapTemplate.startNavigationSession (for: trip)
 }  

CPNavigationSession is a class with which you can display some UI elements that are needed only in navigation mode.


To display the maneuver, you must:


  let maneuver = CPManeuver ()
 maneuver.symbolSet = CPImageSet (
  lightContentImage: icon,
  darkContentImage: darkIcon
 )
 maneuver.instructionVariants = ["ul. Kutateladze"]
 maneuver.initialTravelEstimates = CPTravelEstimates (...)

 self.navigationSession? .upcomingManeuvers = [maneuver]  

Then on the screen of the radio, we get this:



To update the footage to the maneuver, you need:


  let estimates = CPTravelEstimates (...)
 self.navigationSession? .updateEstimates (estimates, for: maneuver)  

It just works!


When the main functionality for the navigator was ready, I decided to show this craft on the internal presentation. The presentation was a success: everyone got excited about the idea to finish, test and launch the navigator as soon as possible.


First of all, we ordered a real head unit with CarPlay support. And here, as they say, the heat has gone.


PIONEER AVH-Z500BT


Provision Profiles


Due to the addition of a new capability-key, profiles need to be regenerated. In normal development, we do not think about it, because Xcode will do everything by itself. But not in the case of a private key.


  Target entitlements.  Signing your add-in com.apple.developer.carplay-maps entitlement to your provisioning profile.  There is no need for additional information.  

It also broke our CI, as for the local distribution of application versions we use an enterprise account to which we did not request permission to develop an application for CarPlay. But that's another story.


Debugging


You can connect to CarPlay via Bluetooth or Lightning. Practice shows that the second method is much more popular. Our Bluetooth radio tape recorder did not know how, so during development we had to use Wi-Fi debug.If you tried it on projects harder than hello world, then you know what kind of hell it is.


And for those who have not tried, I tell:

I assembled the application over the wire to my phone, and only then, having connected the phone to CarPlay, via Wi-Fi, poured it onto the phone and started up for a few minutes.
The application was copied to the phone for about 3 minutes, the application was started for about a minute and only after the launch, the stop at breakpoint was only 15 seconds later.


And then it became very interesting to me why Apple did not do any DevKit (so that Apple-way, it just works and that’s all). Without it, collecting a test stand was not very convenient. So far, once in a couple of weeks, something falls off - you have to remember from the pics what to stick with. It’s good that the admin told me how and why to assemble this stand.


The best framework we ever made


In the end, when everything gathered on the real device, it became clear that the feature "2GIS under CarPlay" is exactly to be. It is time to make beauty.


Problems with viewport


It was necessary to set up a map viewport to draw routes in the area without extra controls, and not just in the middle. In short, to make it look wrong:



And so:



I figured I’ll get some layoutGuide with the current visible area. So that it takes into account both the navigationBar, the view with the route, and the controls on the map. In fact, I got nothing. It is still not clear how to customize the viewport, so we have a hardcode in the code like:


  let routeControlsWidth = self.view.frame.width * 0.48
 let zoomControlWidth = self.view.frame.width * 0.15  

Building a passage not only between two points


In the first release, we decided to take our rubricator made via CPGridTemplate:



Favorites and Home/Jobs via CPListTemplate.



And keyboard search via CPSearchTemplate:



I will not show the code about templates, since it is simple and the documentation is well written about it (at least about something).


However, it is worth mentioning what problems were found when working with them.

CPInterfaceController can in navigation, similar on uikit. i.e.


  self.interfaceController.pushTemplate (listTemplate, animated: true)
 self.interfaceController.presentTemplate (alertTemplate, animated: true)  

But if you try to launch, for example, CPAlertTemplate, you will get an assertion in the logs that CPAlertTemplate can only be modally represented.


It’s not clear why Apple didn’t hide the transcendental logic under the hood without making an interface like:


  self.interfaceController.showTemplate (listTemplate, animated: true)  

It also broke the ability to use CPTemplate heirs, like controllers in UIKit.


When you try, for example, to put your heir in the stack of templates, you will get this:


  Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Unsupported object & lt; YourAwesomeGridTemplate: 0x60000060dce0 & gt;  & lt; identifier: 6CAC7E3B-FE70-43FC-A8B1-8FC39334A61D, userInfo: (null) & gt;  passed to pushTemplate: animated :.  Allowed classes: {(
  CPListTemplate,
  CPGridTemplate,
  CPSearchTemplate,
  CPMapTemplate
 )} ’ 

Testing and Bugs


Testing was done by artemenko-a-a . One of the first bugs that he found, we still can not fix it.


The fact is that when you disconnect the phone from the CarPlay radio tape recorder, we are sporadically beat Watchdog - without explaining the reason. Even syslogs opened, nothing is clear. So if you have an idea how to fix or understand the cause, then you are welcome in the comments.


The next bug was in the same place, but with a special behavior. I wrote above that the CPApplicationDelegate method didDisconnect is called when the phone is disconnected from CarPlay. And in this method we return the map from the screen of the radio tape recorder back to the main application. Imagine how many problems we would have had if this method had not been called at least once out of five.


It became clear that this is a problem of iOS, and not specifically our application, since the entire system believed that it was connected to CarPlay.



I even reported it as a radar (like all the other bugs). I was asked to drop logs with such a profile, but I could not answer support for some time, so they closed the radar.


Once Apple didn’t plan to do anything, the problem had to be avoided on its own, as it was reproduced quite often.


And here I remembered that the lion's share of connections to CarPlay goes through Lightning. This means that the phone is charging at the time of connection, and at the time of shutdown it stops charging. And if so, you can subscribe to the battery status and find out exactly when the phone stopped charging and disconnected from CarPlay.


The scheme is tiny, but we had no choice. We went this way, and everything worked!



Fortunately, this crutch from the code has long been removed: Apple developers fixed everything in one of the iOS releases.


History of two rejects


The first reject was associated with the metadata. The text of the redesign said that our description (not release notes) did not say that we support CarPlay. As you can guess, neither in the review guidelines, nor in the same Google Maps this was not. We did not argue (because it is usually longer than editing the metadata), copied the line from the Release Notes in the Description and waited for a new review.


Second Redget happened because of the list of cities. 2GIS has a very cool feature - full offline mode. This feature shot us in the leg.


When you connect an application without a city set to CarPlay, we don’t show the map, because there’s nothing to show. For this, we and zredzhektili. The solution was simple: an alert without buttons, which says that you need to download the city.



What you can not say


Move the map with gestures


At about the same time, the navigator came out under CarPlay from Google Maps - and there it was possible to move the map with gestures around the screen. Private APIs, I thought, that's obvious! The guys from Google just came from a nearby building and said they need it.After all, the documentation says:


  There is no support for the user.   

However, I still decided to make sure and went Google, although it was almost meaningless, because there were no technical articles about CarPlay Navigation Apps. However, I managed to find something useful and, SUDDENLY, on the Apple website .


In the guidelines, I found a video that says the documentation is blatantly lying. The video shows how the map can still be dragged with gestures. I realized that I did not understand anything, and the only thing left for me was to open CarPlay.framework and revise all .h files.


And lo and behold! I find in CPMapTemplate his delegate CPMapTemplateDelegate, in which there are 3 methods that seem to shout that if you implement them, you can get control of map gestures.


3 methods

/* Called when a pan gesture begins. May not be called when connected to some CarPlay systems.
/
optional public func mapTemplateDidBeginPanGesture (_ mapTemplate: CPMapTemplate)


/* Called when a pan gesture changes. May not be called when connected to some CarPlay systems.
/
optional public func mapTemplate (_ mapTemplate: CPMapTemplate, didUpdatePanGestureWithTranslate translation: CGPoint, velocity: CGPoint)


/* Called when a pan gesture ends. May not be called when connected to some CarPlay systems.
/
optional public func mapTemplate (_ mapTemplate: CPMapTemplate, didEndPanGestureWithVelocity velocity: CGPoint
)


I implemented them and ran the application on the simulator - nothing worked. Not having time to get upset, I realized that the simulator can be of the same quality as the documentation, and assembled on the device. Everything started, fortunately there was no limit!


Fun fact: a CarPlay radio tape recorder needs a quarter of the screen to understand that a pan gesture has begun. I want to note that the UIPanGestureRecognizer only needs 10 points.


UI is not the same on different radio tape recorders


We received support in support: the user gets only one sadest in the search, although there could be more. Strange, I thought, because on all screens fit only one line. Request a screenshot:



And this is completely different from the CPSearchTemplate UI that I showed above. And this should be taken into account when developing, although it is impossible to understand how many cells in the plate below can fit into the screen.


Speed ​​Limit Control


We looked at the statistics and realized that the navigator for CarPlay is used and we must bring it at least to the level of the navigator in the main application. First decided to add a speed limit control. No problem, of course, not done.


Question number one: where to post?


After reviewing the .h files in the CPWindow again, I found a curious layoutGuide:
var mapButtonSafeAreaLayoutGuide: UILayoutGuide


And it turned out to be the right thing. Our control fit in perfectly there:




Question number two: is this generally legal?


The fact is that technically the control is on the base view. And the base view of the documentation can not contain anything but the map:


  The base view is where the map is drawn.It can be used to display other UI elements.  Instead, navigation apps overlay UI elements such as the navigation bar and map buttons.   

But the reviewers missed us in the AppStore, which means that the controls that relate to navigation can still be embedded.


Voice Search




In an amicable way, this feature needed to be done first of all, but we have accumulated a few technical debt tasks that prevented the voice search for CarPlay from being implemented. And this task was not as simple as it seemed.


Problem one: animations. The fact is that in CPVoiceControlTemplate there is no possibility to make standard animations. Animation for speech recognition and search had to be collected frame by frame from the pictures and indicate how much they go on time.


  for i in 1 ... 12 {
  if let image = UIImage (named: "carplay_searching _ \ (i)") {
  images.append (image)
  }
 }

 let image = UIImage.animatedImage (with: images, duration: 0.96)  

It looks like you can guess, not really, but you don’t want to inflate the size of the application.


Problem Two: Access. Alerts for microphone access and speech recognition appear on the phone’s display. I had to write on the display of the radio tape recorder that the user needed to take the phone in hand, give permission, and only then use the navigator on the radio tape recorder. Very convenient!


Right-hand drive cars.


They sent us a screenshot in which the UI of the entire application was turned over!



And, of course, the map viewport remained the same as we did, because no one expected that there is a separate setting for right-hand drive cars. I didn’t find how to get around this “correctly”, but noticed that since our control for speed limits lies in the layoutGuide for the map controls, it moved to the left side.


Ultrafix did not take long. Did it rough, but it works.


  let isLeftWheelCar = self.speedControlViewController.view.frame.origin.x & gt;  self.view.frame.size.width/2.0  

I really hope that there is a right decision, and I just didn’t read it.


I have it all. If suddenly you are going to do your navigator under CarPlay, please note that the documentation and framework are imperfect. The platform is completely new, nobody knows anything, and Apple is not in a hurry to share knowledge.

Source text: How we launched 2GIS under CarPlay and are still unraveling